U.S. GOVERNMENT PRINTING OFFICE

WASHINGTON : 

For sale by the Superintendent of Documents, U.S. Government Printing Office

Internet: bookstore.gpo.gov Phone: toll free (866) 512–1800; DC area (202) 512–1800

Fax: (202) 512–2104 Mail: Stop IDCC, Washington, DC 20402–0001

53–267 PDF 2009

S. HRG. 111–149 

ACQUISITION OF MAJOR WEAPONS SYSTEMS BY 

THE DEPARTMENT OF DEFENSE AND S. 454, 

THE WEAPON SYSTEMS ACQUISITION REFORM 

ACT OF 2009 

HEARING 

BEFORE THE

COMMITTEE ON ARMED SERVICES 

UNITED STATES SENATE 

ONE HUNDRED ELEVENTH CONGRESS 

FIRST SESSION 

MARCH 3, 2009 

Printed for the use of the Committee on Armed Services 

( 

COMMITTEE ON ARMED SERVICES 

CARL LEVIN, Michigan, Chairman 

EDWARD M. KENNEDY, Massachusetts 

ROBERT C. BYRD, West Virginia 

JOSEPH I. LIEBERMAN, Connecticut 

JACK REED, Rhode Island 

DANIEL K. AKAKA, Hawaii 

BILL NELSON, Florida 

E. BENJAMIN NELSON, Nebraska 

EVAN BAYH, Indiana 

JIM WEBB, Virginia 

CLAIRE McCASKILL, Missouri 

MARK UDALL, Colorado 

KAY R. HAGAN, North Carolina 

MARK BEGICH, Alaska 

ROLAND W. BURRIS, Illinois 

JOHN MCCAIN, Arizona 

JAMES M. INHOFE, Oklahoma 

JEFF SESSIONS, Alabama 

SAXBY CHAMBLISS, Georgia 

LINDSEY GRAHAM, South Carolina 

JOHN THUNE, South Dakota 

MEL MARTINEZ, Florida 

ROGER F. WICKER, Mississippi 

RICHARD BURR, North Carolina 

DAVID VITTER, Louisiana 

SUSAN M. COLLINS, Maine 

RICHARD D. DEBOBES, Staff Director 

JOSEPH W. BOWAB, Republican Staff Director 

(II) 

C O N T E N T S 

CHRONOLOGICAL LIST OF WITNESSES 

ACQUISITION OF MAJOR WEAPONS SYSTEMS BY THE DEPARTMENT OF DEFENSE AND

S. 454, THE WEAPON SYSTEMS ACQUISITION REFORM ACT OF 2009 

MARCH 3, 2009

Page 

Sullivan, Michael J., Director, Acquisition and Sourcing Management, Government Accountability Office ............................................................................. 7 

Gansler, Hon. Jacques S., Chairman, Defense Science Board Task Force on 

Industrial Structure for Transformation ............................................................ 19 

Kaminski, Hon. Paul G., Chair, Committee on Pre-Milestone A Systems Engineering, Air Force Studies Board, National Research Council ........................ 25 

Adolph, Charles E. (Pete), Chairman, Defense Science Board Task Force 

on Developmental Test and Evaluation ............................................................. 40 

(III) 

(1) 

ACQUISITION OF MAJOR WEAPONS SYSTEMS 

BY THE DEPARTMENT OF DEFENSE AND

S. 454, THE WEAPON SYSTEMS ACQUISITION 

REFORM ACT OF 2009 

TUESDAY, MARCH 3, 2009 

U.S. SENATE, 

COMMITTEE ON ARMED SERVICES, 

Washington, DC. 

The committee met, pursuant to notice, at 9:36 a.m. in room SH– 

216, Hart Senate Office Building, Senator Carl Levin (chairman) 

presiding. 

Committee members present: Senators Levin, Lieberman, Akaka, 

Bill Nelson, Webb, McCaskill, Udall, Hagan, Begich, Burris, 

McCain, Chambliss, Thune, Martinez, Burr, and Collins. 

Committee staff members present: Richard D. DeBobes, Staff Director, and Leah C. Brewer, Nominations and Hearings Clerk. 

Majority staff members present: Creighton Greene, professional 

staff member; Peter K. Levine, general counsel; John H. Quirk V, 

professional staff member; Arun A. Seraphin, professional staff 

member; and William K. Sutey, professional staff member. 

Minority staff members present: Joseph W. Bowab, Republican 

staff director; Daniel A. Lerner, professional staff member; David 

M. Morriss, minority counsel; Lucian L. Niemeyer, professional 

staff member; and Christopher J. Paul, professional staff member. 

Staff assistants present: Jessica L. Kingston, Brian F. Sebold, 

and Breon N. Wells. 

Committee members’ assistants present: Jay Maroney, assistant 

to Senator Kennedy; Bonni Berge, assistant to Senator Akaka; 

Christopher Caple, assistant to Senator Bill Nelson; Jon Davey, assistant to Senator Bayh; Gordon I. Peterson, assistant to Senator 

Webb; Stephen C. Hedger, assistant to Senator McCaskill; Jennifer 

Barrett, assistant to Senator Udall; Michael Harney, assistant to 

Senator Hagan; David Ramseur, assistant to Senator Begich; 

Brady King, assistant to Senator Burris; Sandra Luff, assistant to 

Senator Sessions; Clyde A. Taylor IV, assistant to Senator 

Chambliss; Jason Van Beek, assistant to Senator Thune; Erskine 

W. Wells III, assistant to Senator Wicker; and Kevin Kane, assistant to Senator Burr. 

OPENING STATEMENT OF SENATOR CARL LEVIN, CHAIRMAN 

Chairman LEVIN. Good morning, everybody. The committee 

meets today to consider the performance of the Department of Defense’s (DOD) acquisition programs at a time when cost growth on 

2 

these programs has reached levels that we cannot afford, including 

consideration of our bill, S. 454, the Weapon Systems Acquisition 

Reform Act of 2009, which Senator McCain and I recently introduced. 

Since the beginning of 2006, nearly half of the DOD’s largest acquisition programs have exceeded the so-called Nunn-McCurdy 

cost-growth standards established by Congress to identify seriously 

troubled programs. 

As Secretary Gates pointed out in his testimony before our committee last month, the list of big-ticket weapon systems that have 

experienced contract or program performance problems spans the 

Services and includes the Air Force tanker, the CSAR–X, the VH– 

71, the Osprey, the Future Combat Systems (FCS), the Armed Reconnaissance Helicopter, the Littoral Combat Ship (LCS), the Joint 

Strike Fighter (JSF), and so on. 

Overall, DOD’s 95 defense acquisition programs, known as Major 

Defense Acquisition Programs (MDAPs), have exceeded their research and develop budgets by an average of 40 percent, seen their 

acquisition costs grow by an average of 26 percent, and experienced 

an average schedule delay of almost 2 years. Last summer, the 

Government Accountability Office (GAO) reported that cost overruns on DOD’s MDAPs now total $295 billion over the original program estimates. That’s true, even though we’ve cut unit quantities 

and reduced performance expectations on many programs, in an effort to hold costs down. 

These cost overruns happen because of fundamental flaws that 

are endemic to our acquisition system. We have a pretty good idea 

of what those flaws are. Major acquisition programs fail because 

DOD: one, continues to rely on unreasonable cost and schedule estimates; two, establishes unrealistic performance expectations; 

three, insists on the use of immature technologies; and four, adopts 

costly changes to program requirements, production quantities, and 

funding levels in the middle of ongoing programs. 

Earlier this year, Under Secretary of Defense for Acquisition 

John T. Young, Jr., wrote a memo in which he sought to explain 

the cost growth on some of DOD’s largest programs. This is what 

his memorandum said: ‘‘A number of programs had a poor foundation and Milestone B, the starting point for major development and 

manufacturing design. . . Fundamentally, these programs moved 

past Milestone B with inadequate foundations built on artificially 

low cost estimates, optimistic schedules and assumptions, immature design or technology, fluid requirements, and other issues.’’ 

Mr. Young then went on to list the flaws of each MDAP. The 

JSF: too little understanding of the design; the FCS: fluid program 

strategy; the V–22: immature technology and Congress reversed 

DOD termination; the C–17: development issues and underfunding; 

the Army’s Family of Medium Tactical Vehicles: design flaws; the 

CH–47F: low estimates and invalid remanufacture assumptions; 

the advanced EHF Satellite: optimistic schedule; the LPD–17: 

flawed lead ship design process and knowledge base; and the F– 

22A: immature, exquisite technology. 

Now, the first two of these programs, JSF and FCS, account for 

almost $80 billion in cost overruns, with average unit costs that 

have already increased by roughly 40 percent each over original 

3 

program estimates, and are likely to rise further. According to 

GAO, both programs were initiated with insufficiently mature technologies and overly optimistic assumptions about system performance. 

With regard to the JSF, GAO reports that initial estimates assumed that commonality between the three variants of the aircraft 

could cut development costs by about 40 percent; however, this 

level of commonality has proven impossible to achieve. Twelve 

years after the program started, three of the JSF’s eight critical 

technologies are still not mature, its production processes are not 

mature, and its designs are still not fully proven and tested. 

With regard to FCS, GAO reports that the estimated lines of 

code needed to support FCS’s software and development are almost 

three times the original assumptions, leading to an increase in software development costs that now approaches $8 billion. Eight 

years after the program started, only 3 of the FCS’s 44 critical 

technologies are fully mature. GAO tells us that the Army has not 

advanced the maturity of 11 critical technologies since 2003, and 

that 2 other technologies, which are central to the Army’s plans, 

are now rated less mature than when the program began. 

This is the price that we have paid for our failure to complete 

needed systems engineering tasks, perform appropriate developmental testing, and build prototypes. Particularly at this time, 

when the Federal budget is under immense strain as a result of the 

economic crisis, we cannot continue this kind of waste and inefficiency. 

That is why Senator McCain and I have introduced the Weapon 

Systems Acquisition Reform Act of 2009. This bill is designed to 

help put MDAPs on a sound footing from the outset by addressing 

program shortcomings in the early phases of the acquisition process. 

In particular, our bill would address unreasonable performance 

requirements by requiring DOD to rebuild its systems engineering 

capability, reestablish the position of director of developmental 

testing, and use the Joint Requirements Oversight Council (JROC) 

to make early tradeoffs between cost schedule and performance requirements. 

Our bill will address unreasonable cost and schedule estimates 

by establishing a new director of independent cost assessment to 

ensure that cost estimates for MDAPs are fair, reliable, and unbiased. 

Our bill will reduce the use of immature technologies by requiring DOD to make greater use of prototypes, including competitive 

prototypes, and requiring the Director of Defense Research and Engineering to periodically review and provide independent assessments of the maturity of critical technologies on major weapon systems. 

Our bill, finally, addresses costly changes in the middle of programs by ensuring, through preliminary design review, that requirements are well understood before a program receives Milestone B approval, by providing an incentive for contractors to improve performance in ongoing programs by developing mechanisms 

to maintain competitive pressure through the program cycle, and 

by tightening the so-called Nunn-McCurdy requirements for underVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00007 Fmt 6633 Sfmt 6633 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
4 

performing programs by providing for the termination of any program that cannot be justified after a complete reexamination and 

revalidation. 

Today, we will hear from two distinguished former Under Secretaries of Defense for Acquisition, Paul Kaminski and Jacques 

Gansler. We will also hear from Pete Adolph, a former DOD Director of Developmental Testing, and Mike Sullivan, the GAO Director 

of Acquisition and Sourcing Management. Each of our witnesses 

has great experience in the area of weapon systems acquisition; 

and, in the course of the last year, each has completed a major report recommending significant improvements and reforms. We all 

look forward to their testimony on these issues. 

I now call on Senator McCain. 

STATEMENT OF SENATOR JOHN MCCAIN 

Senator MCCAIN. Thank you, Mr. Chairman, for today’s hearing, 

and more importantly, your leadership on the bill that is the subject of today’s hearing, the Weapons Acquisition Reform Act of 

2009. I join you in welcoming our expert witnesses today. 

Let me set the overall context of today’s discussion, and I’ll do 

so very simply. A train wreck is coming. Look at the President’s 10- 

year budget and you’ll see an overall decrease in defense spending. 

Unless difficult decisions are made and serious reform measures 

are undertaken, our ability to provide for our national security will 

be, over time, fundamentally compromised. 

Clearly, the endless cycle of runaway costs, prolonged delivery 

schedules, and poor performance in the acquisition of major weapons has, in my view, us mired in a form of unilateral disarmament. 

Since scrutinizing the tanker lease scandal years ago, I’m not sure 

that things are any better. For example, how could DOD award a 

multibillion-dollar contract based on a proposal it later found was 

fundamentally unexecutable? That’s exactly what happened on the 

Navy’s VH–71 program, the program to replace the President’s own 

helicopters. 

Just over the last few years, the VH–71 program has doubled, with an additional cost of $6 billion for 28 aircraft that will likely 

cost taxpayers well over $400 million each. How could DOD laden 

a multibillion-dollar shipbuilding program with so many requirements that the program more than doubled in cost, with DOD basically asleep at the switch? That happened with the Navy’s LCS 

program. At times, the program saw change orders averaging 75 

per week. 

How could a multibillion-dollar program for next-generation 

fighter jets produce planes that are operating below satisfactory 

readiness rates and could end up being too expensive to operate? 

That happened with the Air Force’s F–22 Raptor program. 

How could DOD spend billions for the Army’s biggest trans- formational program, valued at almost $200 billion, only for it to 

be, in many respects, closer to the beginning of development than 

it is to the end? That’s the FCS program. At this point, it’s not 

been clear when, or even if, the information network at the heart 

of the FCS concept can be built. 

On our military satellite program, how could a design flaw recently emerge that will take at least 1 year, and up to $1 billion 

5 

to fix? That’s the Air Force’s Space-Based Infrared System (SBIRS), 

high satellite program. More cost and schedule increases are likely 

there. 

But, to understand the depth and breadth of our acquisition 

problems, one needs to go no further than to look at the status of 

particular programs. Across all the Services, the top 75 programs 

have unfunded cost overruns of at least $295 billion. From 2000, 

the number of MDAPs has increased from 75 to 95, and the cost 

of those programs has doubled from $790 billion to $1.6 trillion, 

leaving unfunded acquisition commitments equal to more than 10 

years’ worth of major weapons procurement funding. 

In other words, in the current fiscal environment we find ourselves, the DOD acquisition plan is unaffordable. In my view, 

meaningful reform is only going to happen if DOD itself decides to 

change, develops an overarching management philosophy, sets up 

clear lines of authority and accountability, brings discipline and 

control over the requirement process, shuts the revolving door, and 

restores the corps of qualified and experienced acquisition and contracting professionals. That’s what this legislation helps to do. 

In this bill, the chairman and I built on previous reform initiatives by focusing on costs and risk. The bill reflects that a key to 

managing defense procurement programs effectively is starting 

them right by requiring key program reviews up front to catch costly design flaws and technology risks before we actually buy them. 

Probably the most aggressive feature of the bill gives DOD a big 

stick, bigger than anything available under current law, to wield 

against the very worst-performing programs. It does so by giving 

DOD additional tools to enforce fair, reliable, and unbiased independent cost estimates with the creation of a new director. Unlike 

merely promulgated DOD instructions, which apply only to new 

programs, that provision will capture chronically-poor performers 

that are in the development pipeline now. 

Mr. Chairman, I don’t want to go all over the features of the bill; 

I want to hear from our witnesses. But, for truly meaningful reform to endure, the commitment to reform must begin with the fiduciaries of the taxpayers dollars within the DOD itself. 

I thank you, Mr. Chairman. I welcome the witnesses. 

[The prepared statement of Senator McCain follows:] 

PREPARED STATEMENT BY SENATOR JOHN MCCAIN

Thank you, Mr. Chairman, for today’s hearing and, more importantly, your leadership on the bill that is the subject of today’s hearing, S.454, ‘‘The Weapons Systems Acquisition Reform Act of 2009.’’ 

I appreciate the opportunity to serve as an original co-sponsor on it with you, and 

I too welcome our expert witnesses today. 

Before I turn to the bill, let me briefly set the overall context of today’s discussion, 

and I’ll do so simply. 

A train wreck is coming, and unless hard decisions are made and serious reform 

measures undertaken, our ability to see to our national security interests will be 

over time fundamentally compromised. 

Let me be clear. The endless cycle of runaway costs, prolonged delivery schedules 

and poor performance we have seen in major weapons has us, in my view, mired 

in a form of unilateral disarmament. 

Since we closely scrutinized the tanker lease scandal years ago, I’m not sure that 

things have gotten much better. For example: 

• How could the DOD have awarded a multibillion contract based on a proposal it later found was ‘‘fundamentally unexecutable.’’ That’s exactly what 

6 

happened on the Navy’s VH–71 program—the program to replace the President’s own helicopters. Just over the last few years, the program has increased by about $6 billion—for aircraft that will likely cost taxpayers well 

over $400 million each. 

How could the DOD have loaded up a multibillion shipbuilding program with so many requirements that the program doubled in cost (by about $400 million) without the DOD really noticing until it was too late? That happened—on the Navy’s 

Littoral Combat Ship (LCS) program. At times, that program saw change orders 

averaging 75 per week. • How could a multibillion dollar program for next generation fighter jets 

have produced planes that may end up being too expensive to operate? That 

too happened—on the F–22 Raptor program. While being the Nation’s most 

expensive fighter aircraft, those jets continue to operate below satisfactory 

reliability rates. 

• How could the DOD have spent billions for the Army’s biggest trans- formational program, valued at about $200 billion, only for it to be (in many 

respects) closer to the beginning of development than it is to the end? 

That’s the Future Combat Systems (FCS) program. At this point, it’s not 

even clear when (or even if) the information network at the heart of the 

FCS concept can be built. 

• On a military satellite program, how could a design flaw have recently 

emerged that will take at least 1 year and up to $1 billion to fix? That’s 

the Air Force’s Space-Based Infrared System High satellite program. More 

cost and schedule increases are likely there. 

We’re supposed to have laws in the books that are supposed to prevent these sorts 

of things from happening. Why didn’t they work? To the person—who is responsible? 

The fact that we’re asking those questions (with billions of dollars of taxpayer 

money at stake) and that we have no answers, lays out what we’re dealing with far 

better than I can possibly describe using facts and figures. 

So, I don’t have to mention that the top 75 programs across all of the Services 

have unfunded cost overruns of at least $295 billion. Or, that (since fiscal year 2000) 

the number of major defense acquisition programs has increased from 75 to 95. Or, 

that within that period the cost of those programs doubled from $790 billion to $1.6 

trillion. Or, that this left unfunded acquisition commitments equal to more than 10 

years worth of major weapons procurement funding. 

I don’t have to talk about how risky developing most of those programs are; or 

the likelihood that they too will balloon in costs; or how much other governmentwide priorities will constrain defense spending going forward. I don’t have to go into 

all that to make the point that the DOD’s acquisition plan—as it currently stands— 

is itself likely unaffordable. 

However one looks at it, the honeymoon is over. 

In my view, meaningful reform is only going to happen if DOD itself decides to 

change. DOD has to: 

• Develop an overarching management philosophy that dictates an overall 

approach to ensuring the timely delivery of major weapons that satisfy the 

needs of the joint warfighter at the most reasonable cost to the taxpayer. 

• Set up clear lines of authority and accountability for managing procurement programs. 

• Bring discipline and control over the requirements process and get out of 

the business of gold-plating programs. 

• Shut the revolving door. While a leavening of experienced DOD procurement officials working for defense contractors (and vice-versa) is healthy, 

the lack of meaningful controls on this revolving door is creating an 

unhealthy tolerance of conflict-of-interest. 

• Restore the corps of qualified and experienced acquisition and contracting 

professionals that DOD had in the 1980s before it gave its functions over 

to contractor/lead systems integrators, thereby letting the fox guard the hen 

house. 

Until and unless administration and DOD leadership do these sorts of things— 

things that set, if you will, a ‘‘command climate’’ that’s conducive to investing the 

taxpayers’ money responsibly—I fear that reform efforts may amount to only rearranging the deck chairs on the Titanic. That’s something that the chairman and I 

have no interest in doing. 

At the end of the day, we in Congress can only give DOD tools that it can use 

to pursue truly lasting solutions that ensure desirable cost/scheduling and performance outcomes in our most complicated, most expensive weapons systems. That’s 

what this bill helps do. 

7 

In this bill, the chairman and I build on previous reform initiatives by focusing 

on costs and risk. The bill reflects that a key to managing defense procurement programs effectively is starting them right—by requiring key program review upfront 

to catch costly design flaws and technology risk before we actually buy the system. 

In so doing, we continue our efforts to move DOD closer towards fixed price-type 

procurement contracting by requiring that technology and integration risk can be 

meted out of a program early. So, by the time a program heads into procurement, 

if a contractor isn’t ready to sign a good fixed price-type contract, the government 

shouldn’t be signing a contract with that company to buy that system at all. There’s 

still too much risk. 

Probably the most aggressive feature of the bill gives DOD a big stick—bigger 

than anything available under current law—to wield against the very worst performing programs. With that provision we intend DOD to, if you will, ‘‘enforce’’ fair, 

reliable, and unbiased cost estimates verified by a new director of independent cost 

estimates, also created by this bill. Unlike rules that DOD recently put in place, 

that provision doesn’t apply to just new programs. It will capture chronically poor 

performers that are in the development pipeline now. 

Another important provision requires DOD to consider a broad range of cost-effective measures to help maximize competition throughout the life of a weapons program. The industry consolidation that occurred after the end of the Cold War went 

too far. Some 50 prime contractors merged into only 6. That’s far too few to support 

a competitive base for our current and future needs. It’s resulted in a serious decline 

in innovation. 

Other provisions in the bill elaborate in the ‘‘starting programs right’’ theme by: 

• Renewing focus on systems engineering early; 

• Requiring the completion of preliminary design reviews before a program 

can move into the development phase; and 

• Strengthening DOD’s developmental testing and evaluation capability. 

Other helpful provisions include those that: 

• Require DOD budget, requirements, and acquisition officials to consult 

each other and make trade-offs between cost, schedule, and performance 

early in the process; and 

• Crack down on the frequent changes to programs, which tend to cause 

many cost increases. 

While this bill is not intended as a panacea to cure all that ails the defense procurement process, it is an important next step in Congress’ continuing efforts to 

help DOD culturally reform the system. 

As I said a moment ago, for meaningful reform to truly endure, the commitment 

to reform must begin with the fiduciaries of the taxpayers’ dollars within the Department itself. 

Thank you, Mr. Chairman. I look forward to hearing from our witnesses today. 

Chairman LEVIN. Senator McCain, we thank you. 

Now we’ll call on our witnesses. First, we’ll call on Michael Sullivan from the GAO. 

Would you please proceed? 

STATEMENT OF MICHAEL J. SULLIVAN, DIRECTOR, ACQUISITION AND SOURCING MANAGEMENT, GOVERNMENT ACCOUNTABILITY OFFICE 

Mr. SULLIVAN. Chairman Levin, Senator McCain, members of the 

committee, I’m pleased to be here today to discuss DOD’s acquisition outcomes and the legislation proposed by this committee to improve them. I’ll make a brief oral statement and ask that my written testimony be placed in the record. 

Chairman LEVIN. It will be. 

Mr. SULLIVAN. We’ve been reporting for years on poor cost and 

schedule outcomes on DOD’s major weapon system acquisitions. As 

the chairman noted, most recently we reported that 95 programs 

in DOD’s current portfolio have grown in cost by $295 billion and 

are, on average, delivered about 21 months late. We believe there 

8 

are problems at the strategic and at the program levels that cause 

these outcomes. 

At the strategic level, DOD’s three systemic processes for building its investment strategy are fragmented and broken. The requirement-setting process, known as the Joint Capabilities and Integration Development System (JCIDS) is stovepiped, it does not 

consider resources, and it approves nearly every proposal that it reviews. The funding process accepts programs with unrealistic cost 

estimates, and does not fully fund their development costs. These 

two processes are poorly integrated, and this poor communication 

leads to unhealthy competition, where too many programs are 

chasing too few dollars. 

Finally, at the program level, the acquisition process initiates 

programs with unreliable cost estimates and without knowledge 

from proper systems engineering analysis to understand each 

weapon system’s requirements and the resources that will be needed to achieve them. These programs move forward with too much 

technology, design, and manufacturing risk as a result. 

DOD understands this and has recently revised its policies to address some of these problems. Its new acquisition policy, for instance, encourages more systems engineering activity earlier in 

programs, competitive prototyping to gain knowledge more quickly 

and to maintain competition, earlier milestone reviews, and steering boards to protect programs against the desire to add more requirements once they’ve started. 

Recent decisions by DOD on some programs have been encouraging, and some of the newer programs appear to have undergone 

more disciplined reviews. 

For many years, there’s been a broad consensus that weapon system acquisition problems are serious and their resolution is overdue. With the Federal budget under increasing strain from the economic crisis facing our Nation, the time to change is now. 

In testimony before this committee last month, Secretary of Defense Gates identified many of the systemic problems associated 

with acquisitions and indicated that efforts are underway to address them. 

We believe that the legislation this committee has proposed will 

help address the toughest problems, and we enthusiastically support it. We believe it precisely targets key problem areas, provides 

much-needed oversight, and provides increased authority and independence to the critical functions of cost estimating and development testing by requiring them to report to the Secretary and to 

Congress. 

Among other things, its provision to require a full inventory of 

DOD’s current systems engineering skills is an excellent beginning 

to rebuilding that sorely-needed capability. Its addition of a termination criterion for Nunn-McCurdy breaches sends a strong signal 

to programs to have realistic cost estimates when they start. 

It is important to state that there is also a need for changes to 

the overall acquisition culture and the incentives it provides. The 

culture should begin to change by resisting the urge to achieve the 

revolutionary, but unachievable, capability in one step by allowing 

technologies to mature in the technology base rather than forcing 

them on the acquisition programs too early, by ensuring that urVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00012 Fmt 6633 Sfmt 6602 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
9 

1GAO, Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major 

Weapon System Program Outcomes, GAO–08–619 (Washington, DC: July 2, 2008), and Defense 

Acquisitions: Perspectives on Potential Changes to DOD’s Acquisition Management Framework, 

GAO–09–295R (Washington, DC: Feb. 27, 2009). 

gent requirements are well-defined and quickly achievable, and by 

instituting shorter, more predictable development cycles. 

These changes will not be easy to make. Tough decisions must 

be made about DOD’s overall portfolio of weapon programs and 

about specific programs; and stakeholders from DOD, the military 

services industry, and Congress will have to play a constructive 

role in this decisionmaking. We see the proposed legislation discussed here today as a very healthy step in that direction. 

Mr. Chairman, this concludes my statement. I’ll wait to answer 

questions. 

[The prepared statement of Mr. Sullivan follows:] 

PREPARED STATEMENT BY MICHAEL J. SULLIVAN

Mr. Chairman and members of the committee: I am pleased to be here today to 

discuss the Department of Defense’s (DOD) management of its major weapon system acquisitions—an area that has been on the Government Accountability Office’s 

(GAO) high-risk list since 1990. Prior to and since that time, Congress and DOD 

have continually explored ways to improve acquisition outcomes without significant 

results. While the technological sophistication of DOD weapon systems is unparalleled, major weapon programs continue to cost more, take longer to complete, and 

deliver fewer quantities and capabilities than originally planned. Last year we reported that the cumulative cost growth in DOD’s portfolio of 95 major defense acquisition programs was $295 billion from first estimates and the average delay in delivering promised capabilities to the warfighter was 21 months. Clearly, some problems are to be expected in developing weapon systems given the technical risks and 

complexities involved. However, all too often we have found that cost and schedule 

problems are rooted in poor planning, execution, and oversight. 

Investment in weapon systems is now at its highest level in two decades, and 

DOD plans to spend more than $357 billion over the next 5 years on major defense 

acquisition programs. Effective management of this substantial investment is critical as competition for funding has increased dramatically within the department 

and across the government. DOD faces a number of fiscal pressures: the ongoing 

military campaigns in Afghanistan and Iraq, rising personnel costs, and the rebuilding and modernization of the force. In addition, the economic and fiscal crises now 

facing the Nation have required unprecedented spending by the Federal Government, and budget deficits are projected to remain high for many years to come. At 

a time when the Federal budget is strained by spending needs for a growing number 

of national priorities, it is imperative that DOD get the best value for every dollar 

it invests in weapon system programs. Every dollar wasted during the development 

and acquisition of weapon systems is money not available for other priorities within 

DOD and elsewhere in the government. 

Today, I will discuss: (1) the systemic problems that have contributed to cost, 

schedule, and performance problems in DOD’s acquisition of major weapon systems; 

(2) recent actions the department has taken to address these problems; (3) our observations on the committee’s proposed acquisition reform legislation; and (4) steps 

that Congress and the department need to take to improve the future performance 

of acquisition programs. The statement includes findings from our July 2008 report 

on a knowledge-based funding approach and February 2009 report on potential 

changes to DOD’s acquisition management framework.1 It also draws from our extensive body of work on DOD’s acquisition of weapon systems. A list of our key products is provided at the end of this statement. This work was conducted in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence 

to provide a reasonable basis for our findings and conclusions based on our audit 

objectives. We believe that the evidence obtained provides a reasonable basis for our 

findings and conclusions based on our audit objectives. 

10 

2GAO, Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD’s Acquisition Outcomes, GAO–07–388 (Washington, DC: Mar. 30, 

2007). 3GAO, Defense Acquisitions: DOD’s Requirements Determination Process Has Not Been Effective in Prioritizing Joint Capabilities, GAO–08–1060 (Washington, DC: Sept. 25, 2008). 

FRAGMENTED INVESTMENT DECISIONMAKING, UNEXECUTABLE PROGRAMS, AND LACK OF

ACCOUNTABILITY UNDERLIE POOR ACQUISITION OUTCOMES

Over the past several years our work has highlighted a number of underlying systemic causes for cost growth and schedule delays at both the strategic and program 

levels. At the strategic level, DOD’s processes for identifying warfighter needs, allocating resources, and developing and procuring weapon systems—which together define DOD’s overall weapon system investment strategy—are fragmented. As a result, DOD fails to effectively address joint warfighting needs and commits to more 

programs than it has resources for, thus creating unhealthy competition for funding. 

At the program level, a military Service typically establishes and DOD approves a 

business case containing requirements that are not fully understood and cost and 

schedule estimates that are based on overly optimistic assumptions rather than on 

sufficient knowledge. Once a program begins, it too often moves forward with inadequate technology, design, testing, and manufacturing knowledge, making it impossible to successfully execute the program within established cost, schedule, and performance targets. Furthermore, DOD officials are rarely held accountable for poor 

decisions or poor program outcomes. 

DOD Lacks an Integrated Approach to Balance Weapon System Investments 

At the strategic level, DOD largely continues to define warfighting needs and 

make investment decisions on a Service-by-Service and individual platform basis, 

using fragmented decisionmaking processes. This approach makes it difficult for the 

department to achieve a balanced mix of weapon systems that are affordable and 

feasible and that provide the best military value to the joint warfighter. In contrast, 

we have found that successful commercial enterprises use an integrated portfolio 

management approach to focus early investment decisions on products collectively 

at the enterprise level and ensure that there is a sound basis to justify the commitment of resources.2 By following a disciplined, integrated process—during which the 

relative pros and cons of competing product proposals are assessed based on strategic objectives, customer needs, and available resources, and where tough decisions 

about which investments to pursue and not to pursue are made—companies minimize duplication between business units, move away from organizational stovepipes, 

and effectively support each new development program they commit to. To be effective, integrated portfolio management must have strong, committed leadership; empowered portfolio managers; and accountability at all levels of the organization. 

DOD determines its capability needs through the Joint Capabilities and Integration Development System (JCIDS). While JCIDS provides a framework for reviewing and validating needs, it does not adequately prioritize those needs from a joint, 

departmentwide perspective; lacks the agility to meet changing warfighter demands; 

and validates almost all of the capability proposals that are submitted. We recently 

reviewed JCIDS documentation related to new capability proposals and found that 

most—almost 70 percent—were sponsored by the military Services with little involvement from the joint community, including the combatant commands, which are 

responsible for planning and carrying out military operations.3 Because DOD also 

lacks an analytic approach to determining the relative importance of the capabilities 

needed for joint warfighting, all proposals appear to be treated as equal priorities 

within the JCIDS process. By continuing to rely on capability needs defined primarily by the Services, DOD may be losing opportunities for improving joint 

warfighting capabilities and reducing the duplication of capabilities in some areas. 

The JCIDS process has also proven to be lengthy and cumbersome—taking on average up to 10 months to validate a need—thus undermining the department’s efforts 

to effectively respond to the needs of the warfighter, especially those needs that are 

near term. Furthermore, the vast majority of capability proposals that enter the 

JCIDS process are validated or approved without accounting for the resources or 

technologies that will be needed to acquire the desired capabilities. Ultimately, the 

process produces more demand for new weapon system programs than available resources can support. 

The funding of proposed programs takes place through a separate process, the department’s Planning, Programming, Budgeting, and Execution (PPBE) system, 

which is not synchronized with JCIDS. While JCIDS is a continuous, need-driven 

process that unfolds in response to capability proposals as they are submitted by 

sponsors, PPBE is a calendar-driven process comprising phases occurring over a 2- 

11 

4GAO–08–619. 

year cycle, which can lead to resource decisions for proposed programs that may 

occur several years later. In addition, because PPBE is structured by military Service and defense programs and not by the joint capability areas being used in JCIDS, 

it is difficult to link resources to capabilities. The PPBE process also largely allocates resources based on historical trends rather than on a strategic basis. Service 

shares of the overall budget have remained relatively static for decades, even 

though DOD’s strategic environment and warfighting needs have changed dramatically in recent years. Because DOD’s programming and budgeting reviews occur at 

the back end of the PPBE process—after the Services have developed their budgets—it is difficult and disruptive to make changes, such as terminating programs 

to pay for new, higher-priority programs. 

We recently reviewed the impact of the PPBE process on major defense acquisition programs and found that the process does not produce an accurate picture of 

the department’s resource needs for weapon system programs, in large part because 

it allows too many programs to go forward with unreliable cost estimates and without a commitment to fully fund them.4 The cost of many of the programs we reviewed exceeded the funding levels planned for and reflected in the Future Years 

Defense Program (FYDP)—the department’s long-term investment strategy (see fig. 

1). DOD’s failure to balance its needs with available resources promotes an 

unhealthy competition for funding that encourages sponsors of weapon system programs to pursue overly ambitious capabilities and underestimate costs to appear affordable. Rather than limit the number and size of programs or adjust requirements, DOD opts to push the real costs of programs to the future. With too many 

programs underway for the available resources and high cost growth occurring in 

many programs, the department must make up for funding shortfalls by shifting 

funds from one program to pay for another, reducing system capabilities, cutting 

procurement quantities, or in rare cases terminating programs. Such actions not 

only create instability in DOD’s weapon system portfolio, they further obscure the 

true future costs of current commitments, making it difficult to make informed investment decisions. 

12 

Initiating Programs with Inadequate Knowledge of Requirements and Resources 

Often Results in Poor Outcomes 

At the program level, the key cause of poor outcomes is the approval of programs 

with business cases that contain inadequate knowledge about requirements and the 

resources—funding, time, technologies, and people—needed to execute them. Our 

work in best practices has found that an executable business case for a program 

demonstrated evidence that: (1) the identified needs are real and necessary and that 

they can best be met with the chosen concept; and (2) the chosen concept can be 

developed and produced within existing resources. Over the past several years, we 

have found no evidence of the widespread adoption of such an approach for major 

acquisition programs in the department. Our annual assessments of major weapon 

systems have consistently found that the vast majority of programs began system 

development without mature technologies and moved into system demonstration 

without design stability. 

The chief reason for these problems is the encouragement within the acquisition 

environment of overly ambitious and lengthy product developments—sometimes referred to as revolutionary or big bang acquisition programs—that embody too many 

technical unknowns and not enough knowledge about the performance and production risks they entail. The knowledge gaps are largely the result of a lack of early 

and disciplined systems engineering analysis of a weapon system’s requirements 

prior to beginning system development. Systems engineering translates customer 

needs into specific product requirements for which requisite technological, software, 

engineering, and production capabilities can be identified through requirements 

analysis, design, and testing. Early systems engineering provides the knowledge a 

product developer needs to identify and resolve performance and resource gaps before product development begins by either reducing requirements, deferring them to 

the future, or increasing the estimated cost for the weapon system’s development. 

Because the government often does not perform the proper upfront requirements 

analysis to determine whether the program will meet its needs, significant contract 

cost increases can and do occur as the scope of the requirements changes or becomes 


303ful1.eps
13 

5GAO, Best Practices: Increased Focus on Requirements and Oversight Needed to Improve 

DOD’s Acquisition Environment and Weapon System Quality, GAO–08–294 (Washington, DC: 

Feb. 1, 2008). 6GAO–08–619. 

7The position of Under Secretary of Defense for Acquisition was established in 1986 and the 

title was subsequently changed to the Under Secretary of Defense for Acquisition, Technology, 

and Logistics in 2000. Since 1986, there have been 11 Under Secretaries. 

better understood by the government and contractor. Not only does DOD not conduct disciplined systems engineering prior to the beginning of system development, 

it has allowed new requirements to be added well into the acquisition cycle. We 

have reported on the negative impact that poor systems engineering practices have 

had on several programs, such as the Global Hawk Unmanned Aircraft System,

F–22A, Expeditionary Fighting Vehicle, and Joint Air-to-Surface Standoff Missile.5

With high levels of uncertainty about requirements, technologies, and design, program cost estimates and related funding needs are often understated, effectively setting programs up for cost and schedule growth. We recently assessed the service and 

independent cost estimates for 20 major weapon system programs and found that 

while the independent estimates were somewhat higher, both estimates were too 

low in most cases.6 In some of the programs we reviewed, cost estimates have been 

off by billions of dollars. For example, the initial service estimate for the development of the Marines’ Expeditionary Fighting Vehicle was about $1.1 billion. The department’s Cost Analysis and Improvement Group (CAIG) estimated the development cost of the program to be $1.4 billion, but development costs for the program 

are now expected to be close to $3.6 billion. In the case of the Future Combat System (FCS), the Army’s initial estimate for the development cost was about $20 billion, while CAIG’s estimate was $27 billion. The department began the program 

using the program office’s estimate of $20 billion, but development costs for the FCS 

are now estimated to be $28 billion and the program is still dealing with significant 

technical risk. Estimates this far off the mark do not provide the necessary foundation for sufficient funding commitments and realistic long-term planning. 

The programs we reviewed frequently lacked the knowledge needed to develop realistic cost estimates. For example, program Cost Analysis Requirements Description documents—used to build the program cost estimate—often lack sufficient detail about planned program content for developing sound cost estimates. Without 

this knowledge, cost estimators must rely heavily on parametric analysis and assumptions about system requirements, technologies, design maturity, and the time 

and funding needed. A cost estimate is then usually presented to decisionmakers as 

a single, or point, estimate that is expected to represent the most likely cost of the 

program but provides no information about the range of risk and uncertainty or 

level of confidence associated with the estimate. 

Lack of Accountability for Making Weapon System Decisions Hinders Achieving Successful Outcomes 

DOD’s requirements, resource allocation, and acquisition processes are led by different organizations, thus making it difficult to hold any one person or organization 

accountable for saying no to a proposed program or for ensuring that the department’s portfolio of programs is balanced. DOD’s 2006 Defense Acquisition Performance Assessment study observed that these processes are not connected organizationally at any level below the Deputy Secretary of Defense and concluded that this 

weak structure induces instability and inhibits accountability. Furthermore, a 

former Under Secretary of Defense for Acquisitions, Technology, and Logistics has 

stated that weapon system investment decisions are a shared responsibility in the 

department and, therefore, no one individual is accountable for these decisions. Frequent turnover in leadership positions in the department exacerbates the problem. 

The average tenure, for example, of the Under Secretary of Defense for Acquisition, 

Technology, and Logistics over the past 22 years has been only about 20 months.7

When DOD’s strategic processes fail to balance needs with resources and allow 

unsound, unexecutable programs to move forward, program managers cannot be 

held accountable when the programs they are handed already have a low probability 

of success. Program managers are also not empowered to make go or no-go decisions, have little control over funding, cannot veto new requirements, and have little 

authority over staffing. At the same time, program managers frequently change during a program’s development, making it difficult to hold them accountable for the 

business cases that they are entrusted to manage and deliver. 

The government’s lack of control over and accountability for decisionmaking is 

further complicated by DOD’s growing reliance on technical, business, and procurement expertise supplied by contractors. This reliance may reach the point where the 

foundation upon which decisions are based may be largely crafted by individuals 

14 

who are not employed by the government, who are not bound by the same rules governing their conduct, and who are not required to disclose any financial or other 

personal interests they may have that conflict with the responsibilities they have 

performing contract tasks for DOD. For example, while the total planned commitments to major acquisition programs have doubled over recent years, the size of the 

department’s systems engineering workforce has remained relatively stable, leading 

program offices to rely more on contractors for systems engineering support. Further, in systems development, DOD typically uses cost-reimbursement contracts in 

which it generally pays the reasonable, allocable, and allowable costs incurred for 

the contractor’s best efforts, to the extent provided by the contract. The use of these 

contracts may contribute to the perpetuation of an acquisition environment that 

lacks incentives for contractors to follow best practices and keep costs and schedules 

in check. 

RECENT DOD POLICY CHANGES COULD IMPROVE FUTURE PERFORMANCE OF WEAPON

SYSTEM PROGRAMS

The department understands many of the problems that affect acquisition programs and has recently taken steps to remedy them. It has revised its acquisition 

policy and introduced several initiatives based in part on direction from Congress 

and recommendations from GAO that could provide a foundation for establishing 

sound, knowledge-based business cases for individual acquisition programs. However, to improve outcomes, DOD must ensure that its policy changes are consistently implemented and reflected in decisions on individual programs—not only new 

program starts but also ongoing programs as well. In the past, inconsistent implementation of existing policy has hindered DOD’s efforts to execute acquisition programs effectively. Moreover, while policy improvements are necessary, they may be 

insufficient unless the broader strategic issues associated with the department’s 

fragmented approach to managing its portfolio of weapon system investments are 

also addressed. 

In December 2008, DOD revised its policy governing major defense acquisition 

programs in ways intended to provide key department leaders with the knowledge 

needed to make informed decisions before a program starts and to maintain disciplined development once it begins. The revised policy recommends the completion 

of key systems engineering activities before the start of development, includes a requirement for early prototyping, establishes review boards to identify and mitigate 

technical risks and evaluate the impact of potential requirements changes on ongoing programs, and incorporates program manager agreements to increase leadership 

stability and management accountability. The policy also establishes early milestone 

reviews for programs going through the pre—systems acquisition phase. In the past, 

DOD’s acquisition policy may have encouraged programs to rush into systems development without sufficient knowledge, in part, because no formal milestone reviews 

were required before system development. If implemented, these policy changes 

could help programs replace risk with knowledge, thereby increasing the chances of 

developing weapon systems within cost and schedule targets while meeting user 

needs. Some aspects of the policy were first pilot-tested on selected programs, such 

as the Joint Light Tactical Vehicle program, and indications are that these programs are in the process of acquiring the requisite knowledge before the start of 

systems development. Some key elements of the department’s new acquisition policy 

include: 

• a new materiel development decision as a starting point for all programs 

regardless of where they are intended to enter the acquisition process, 

• a more robust Analysis of Alternatives (AOA) to assess potential materiel 

solutions that address a capability need validated through JCIDS, 

• a cost estimate for the proposed solution identified by the AOA, 

• early program support reviews by systems engineering teams, 

• competitive prototyping of the proposed system or key system elements 

as part of the technology development phase, 

• certifications for entry into the technology development and system development phases (as required by congressional legislation), 

• preliminary design review (PDR) that may be conducted before the start 

of systems development, and 

• configuration steering boards to review all requirements and technical 

changes that have potential to affect cost and schedule. 

As part of its strategy for enhancing the roles of program managers in major 

weapon system acquisitions, the department has established a policy that requires 

formal agreements among program managers, their acquisition executives, and the 

user community setting forth common program goals. These agreements are inVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00018 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
15 

8 10 U.S.C. § 2433(a)(5) requires the Secretary of Defense to report to Congress when a program’s acquisition unit cost increases by at least 25 percent over the current baseline estimate 

or increases by at least 50 percent over the original baseline estimate. 9 National Research Council, Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Systems Acquisition (Washington, DC: February 2008). 

tended to be binding and to detail the progress the program is expected to make 

during the year and the resources the program will be provided to reach these goals. 

DOD also requires program managers to sign tenure agreements so that their tenure will correspond to the next major milestone review closest to 4 years. The department acknowledges that any actions taken to improve accountability must be 

based on a foundation whereby program managers can launch and manage programs toward successful performance, rather than focusing on maintaining support 

and funding for individual programs. DOD acquisition leaders have also stated that 

any improvements to program managers’ performance depend on the department’s 

ability to promote requirements and resource stability over weapon system investments. 

Over the past few years, DOD has also been testing portfolio management approaches in selected capability areas—command and control, net-centric operations, 

battlespace awareness, and logistics—to facilitate more strategic choices for resource 

allocation across programs. The department recently formalized the concept of capability portfolio management, issuing a directive in 2008 that established policy and 

assigned responsibilities for portfolio management. The directive established nine 

joint capability area portfolios, each to be managed by civilian and military co-leads. 

While the portfolios have no independent decisionmaking authority over requirements determination and resource allocation, according to some DOD officials, they 

provided key input and recommendations in this year’s budget process. However, 

without portfolios in which managers have authority and control over resources, the 

department is at risk of continuing to develop and acquire systems in a stovepiped 

manner and of not knowing if its systems are being developed within available resources. 

OBSERVATIONS ON PROPOSED ACQUISITION REFORM LEGISLATION

Overall, we believe that the legislative initiatives being proposed by the committee have the potential, if implemented, to lead to significant improvements in 

DOD’s management of weapon system programs. Several of the initiatives—including the increased emphasis on systems engineering and developmental testing, the 

requirement for earlier PDRs, and the strengthening of independent cost estimates 

and technology readiness assessments—could instill more discipline into the front 

end of the acquisition process when it is critical for programs to gain knowledge. 

Establishing a termination criterion for Nunn-McCurdy cost breaches could help 

prevent the acceptance of unrealistic cost estimates as a foundation for starting programs.8 Having greater involvement by the combatant commands in determining requirements and requiring greater consultation between the requirements, budget, 

and acquisition processes could help improve the department’s efforts to balance its 

portfolio of weapon system programs. In addition, several of the proposals as currently drafted will codify what DOD policy already calls for, but are not being implemented consistently in weapon programs. 

Section 101: Systems Engineering Capabilities 

Requires DOD to: (1) assess the extent to which the department has in place the 

systems engineering capabilities needed to ensure that key acquisition decisions are 

supported by a rigorous systems analysis and systems engineering process; and (2) 

establish organizations and develop skilled employees to fill any gaps in such capabilities. 

The lack of disciplined systems engineering analysis conducted prior to starting 

system development has been a key factor contributing to poor acquisition outcomes. 

Systems engineering activities—requirements analysis, design, and testing—are 

needed to ensure that a weapon system program’s requirements are achievable and 

designable given available resources, such as technologies. In recent years, DOD has 

taken steps to improve its systems engineering capabilities by establishing a Systems and Software Engineering Center of Excellence and publishing guidance to assist the acquisition workforce in the development of systems engineering plans, education, and training. However, as the National Research Council recently reported, 

DOD’s systems engineering capabilities have declined over time and shifted increasingly to outside contractors.9 A comprehensive assessment to determine what sysVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00019 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
16 

10Defense Science Board, Report on Developmental Test & Evaluation (Washington, DC: May 

2008). 

tems engineering capabilities are in place and what capabilities are needed, as recommended in the proposed legislation, is a critical first step in enhancing the function of systems engineering in DOD acquisitions. At the same time, it will be important for DOD to implement steps to ensure systems engineering is applied in the 

right way and at the right time. 

Section 102: Developmental Testing 

Requires DOD to reestablish the position of Director of Developmental Test and 

Evaluation and requires the Services to assess and address any shortcomings in 

their developmental testing organizations and personnel. 

Robust developmental testing efforts are an integral part of the systems development process. They help to identify, evaluate, and reduce technical risks, and indicate whether the design solution is on track to satisfy the desired capabilities. As 

the Defense Science Board reported in 2008, developmental testing in weapon system programs needs to be improved.10 We believe that developmental testing would 

be strengthened by a formal elevation of its role in the acquisition process and the 

reestablishment of a Director of Developmental Test and Evaluation position. Furthermore, requiring the Director to prepare an annual report for Congress summarizing DOD’s developmental test and evaluation activities would provide more accountability. We also agree that the military Services should be required to assess 

their respective developmental testing entities and address any shortcomings. This 

action would help ensure that the Services have the knowledge and capacity for effective developmental test efforts. 

Section 103: Technical Maturity Assessments 

Makes it the responsibility of the Director of Defense Research and Engineering 

(DDR&E) to periodically review and assess the technological maturity of critical 

technologies used in major defense acquisition programs. 

Ensuring that programs have mature technology before starting systems development is critical to avoiding cost and schedule problems, yet for many years we have 

reported that a majority of programs go forward with immature technologies and 

experience significant cost growth. Legislation enacted by Congress in 2006, requiring DOD to certify that the technology in a program has been demonstrated in a 

relevant environment before it receives approval to start system development, has 

begun to help address this problem. Since the legislation was enacted, DOD has 

asked the DDR&E to conduct independent reviews of technology readiness assessments for system development milestone decisions. Although DDR&E reviews are 

advisory in nature, we have seen reviews that have pushed programs to do more 

to demonstrate technology maturity. The improvements that this proposed legislation, as currently written, is intended to bring about may already be occurring in 

DOD. Congress, however, may wish to consider requiring the DDR&E to conduct 

technology readiness reviews not just periodically, but for all major defense acquisition programs, and whether or not DDR&E has the capacity and resources to effectively conduct technology assessments. 

Section 104: Independent Cost Assessment 

Establish a Director of Independent Cost Assessment to ensure that cost estimates for major defense acquisition programs are fair, reliable, and unbiased. 

Within DOD, the CAIG is the organization responsible for conducting independent 

costs estimates for major defense acquisition programs. The CAIG reports to the department’s Director of Program Analysis and Evaluation, but its principal customer 

is the Under Secretary of Defense for Acquisition, Technology, and Logistics. We believe that establishing an independent assessment office that reports directly to the 

Secretary or Deputy Secretary of Defense and to Congress—similar to the Office of 

the Director of Operation Test and Evaluation—would more fully integrate cost estimating with the acquisition management framework and provide an increased level 

of accountability. We see no reason why CAIG should not form the basis of the proposed organization. Congress may also wish to consider appointing the Director for 

a time-certain term and making the Director responsible for prescribing cost-estimating policy and guidance and for preparing an annual report summarizing cost 

estimates for major acquisition programs. Ultimately, however, improved cost estimating will only occur if there is a better foundation for planning and acquiring 

weapon system programs—one that promotes well-defined requirements, is knowledge-based and informed by disciplined systems engineering, requires mature technology, and adheres to shorter development cycle times. 

17 

11GAO–07–388. 

Section 105: Role of Combatant Commanders 

Requires the Joint Requirements Oversight Council (JROC) to seek and consider 

input from the commanders of the combatant commands in identifying joint military 

requirements. 

Requirements determination in DOD, particularly for major weapon system programs, continues to be driven largely by the military Services. Studies by the Defense Science Board, Center for Strategic and International Studies, and others have 

revealed that although the combatant commands—which are responsible for planning and executing military missions—are the principal joint warfighting customer 

in DOD, they have played a limited role in determining requirements. Currently, 

the JROC is doing more to seek out and consider input from the combatant commands through regular trips and meetings to discuss capability needs and 

resourcing issues. However, many of the combatant commands do not believe that 

their needs, which are reflected through the Integrated Priority List process, are 

sufficiently addressed through the department’s JCIDS process. For the combatant 

commands to meet this proposed legislative mandate and have more influence in establishing requirements, DOD should consider providing the combatant commands 

with additional resources to establish robust analytical capabilities for identifying 

and assessing their capability needs. Ultimately, the department must better 

prioritize and balance the needs of the military Services, combatant commands, and 

other defense components, and be more agile in responding to near-term capability 

needs. 

Section 201: Trade-offs of Cost, Schedule, and Performance 

Requires consultation between the budget, requirements, and acquisition processes to ensure the consideration of trade-offs between cost, schedule, and performance early in the process of developing major weapon systems. 

As currently structured, DOD’s budget, requirements, and acquisition processes 

do not operate in an integrated manner. The function and timing of the processes 

are not sufficiently synchronized, and the decisionmakers for each process are motivated by different incentives. These weaknesses have contributed to the development of a portfolio with more programs than available resources can support and 

programs that launch into system development without executable business cases. 

We have recommended that the department establish an enterprisewide portfolio 

management approach to weapon system investment decisions that integrates the 

determination of joint warfighting needs with the allocation of resources, and cuts 

across the Services by functional or capability area.11 To ensure the success of such 

an approach, we believe that the department should establish a single point of accountability with the authority, responsibility, and tools to implement portfolio management effectively. 

Section 202: Preliminary Design Review 

Require the completion of a PDR and a formal post-PDR assessment before a 

major defense acquisition program receives approval to start system development. 

We have found that a key deliverable in a knowledge-based acquisition process 

is the preliminary design of the proposed solution based on a robust systems engineering assessment prior to making a large financial commitment to system development. Early systems engineering provides the knowledge needed by a developer 

to identify and resolve gaps, such as overly optimistic requirements that cannot be 

met with current resources, before product development begins. Consequently, DOD 

would have more confidence that a particular system could successfully proceed into 

a detailed system development phase and meet stated performance requirements 

within cost, schedule, risk, and other relevant constraints. The recently revised 

DOD acquisition policy places an increased emphasis on programs planning for PDR 

prior to the start of system development but does not go as far as making it a requirement to do so. We support any effort to add controls to the acquisition process 

to ensure that timely and robust systems engineering is conducted before major investment decisions, such as the approval to start system development, are made. 

Section 203: Life-Cycle Competition 

Require DOD to adopt measures recommended by the 2008 Defense Science Board 

Task Force on Defense Industrial Structure for Transformation—such as competitive prototyping, dual sourcing, open architectures, periodic competitions for subsystem upgrades, and licensing of additional suppliers—to maximize competition 

throughout the life of a program. 

18 

12GAO, Defense Acquisitions: Better Weapon Program Outcomes Require Discipline, Accountability, and Fundamental Changes in the Acquisition Environment, GAO–08–782T (Washington, 

DC: June 3, 2008). 

We have reported in the past on the problem of diminishing competition and the 

potential benefits of more competition.12 In discussing the environment that leads 

to poor acquisition outcomes, we have noted that changes within the defense supplier base have added pressure to this environment. We noted that in 2006, a DODcommissioned study found that the number of fully competent prime contractors 

competing for programs had fallen from more than 20 in 1985 to only 6, and that 

this has limited DOD’s ability to maximize competition in order to reduce costs and 

encourage innovation. However, avenues exist for reducing costs through competition. For example, we reported that although continuing an alternate engine program for the Joint Strike Fighter would cost significantly more in development costs 

than a sole-source program, it could, in the long run, reduce overall life-cycle costs 

and bring other benefits. 

Section 204: Nunn-McCurdy Breaches 

Requires that a major defense acquisition program that experiences a critical 

Nunn-McCurdy cost breach be terminated unless the Secretary of Defense certifies 

that: (1) continuing the program is essential to national security and the program 

can be modified to proceed in a cost-effective manner; and (2) the program receives 

a new milestone approval prior to the award of any new or modified contract extending the scope of the program. 

In order for DOD to improve its program outcomes, realistic cost estimates must 

be required when programs are approved for development initiation. DOD often 

underestimates costs in large part because of a lack of knowledge and overly optimistic assumptions about requirements and critical technologies. This underestimation is also influenced by DOD’s continuing failure to balance its needs with 

available resources, which promotes unhealthy competition among programs and encourages programs to overpromise on performance capabilities and underestimate 

cost. This false optimism is reinforced by an acquisition environment in which there 

are few ramifications for cost growth and delays. Only in very rare instances have 

programs been terminated for poor performance. When DOD consistently allows unsound, unexecutable programs to begin with few negative ramifications for poor outcomes, accountability suffers. As section 204 proposes, the strengthening of the 

Nunn-McCurdy provision—by including the potential termination of programs that 

experience critical cost growth—could facilitate a change in DOD’s behavior by preventing the acceptance of unrealistic cost estimates as a foundation for program initiation and placing more accountability on senior DOD leadership for justifying program continuation. Programs may thus be forced to be more candid and upfront 

about potential costs, risks, and funding needs, and the likelihood of delivering a 

successful capability to the warfighter at the cost and in the time promised may 

grow. 

Section 205: Organizational Conflicts of Interest 

Prohibits systems engineering contractors from participating in the development 

or construction of major weapon systems on which they are advising DOD, and requires tightened oversight of organizational conflicts of interest by contractors in the 

acquisition of major weapon systems. 

The defense industry has undergone significant consolidation in recent years 

which has resulted in a few large, vertically integrated prime contractors. This consolidation creates the potential for organizational conflicts of interest where, for example, one business unit of a large company may be asked to provide systems engineering work on a system being produced by another unit of the same company. As 

the Defense Science Board has recognized, these conflicts of interest may lead to impaired objectivity, which may not be mitigated effectively through techniques such 

as erecting a firewall between the employees of the two units. While the Federal 

Acquisition Regulation currently covers some cases of potential organizational conflicts of interest involving the systems engineering function, there may be a need 

for additional coverage in this area. In general, we would support efforts to enhance 

the oversight of potential organizational conflicts of interest, particularly in the current environment of a heavily consolidated defense industry. 

Section 206: Acquisition Excellence 

Establishes an annual awards program to recognize individuals and teams that 

make significant contributions to the improved cost, schedule, and performance of 

defense acquisition programs. 

19 

We support the creation of an annual awards program to recognize individuals 

and teams for improving the cost, schedule, and performance of defense acquisition 

programs. We have reported that meaningful and lasting reform will not be 

achieved until the right incentives are established and accountability is bolstered at 

all levels of the acquisition process. The need for incentives emerged as a significant 

issue in our recent discussions with acquisition experts examining potential changes 

to the acquisition processes enumerated in last year’s defense authorization act. The 

discussions revealed that those changes may not achieve the desired improvement 

in acquisition outcomes unless they are accompanied by changes in the overall acquisition environment and culture, and the incentives they provide for success. 

CONCLUDING OBSERVATIONS ON WHAT REMAINS TO BE DONE

A broad consensus exists that weapon system problems are serious and that their 

resolution is overdue. With the Federal budget under increasing strain from the Nation’s economic crisis, the time for change is now. DOD is off to a good start with 

the recent revisions to its acquisition policy, which, if implemented properly, should 

provide a foundation for establishing sound, knowledge-based business cases before 

launching into development and for maintaining discipline after initiation. The new 

policy will not work effectively, however, without changes to the overall acquisition 

environment. Resisting the urge to achieve the revolutionary but unachievable capability, allowing technologies to mature in the science and technology base before 

bringing them onto programs, ensuring that requirements are well-defined and doable, and instituting shorter development cycles would all make it easier to estimate 

costs accurately, and then predict funding needs and allocate resources effectively. 

But these measures will succeed only if the department uses an incremental approach. Constraining development cycle times to 5 or 6 years will force more manageable commitments, make costs and schedules more predictable, and facilitate the 

delivery of capabilities in a timely manner. 

Acquisition problems are also likely to continue until DOD’s approach to managing its weapon system portfolio: (1) prioritizes needs with available resources, 

thus eliminating unhealthy competition for funding and the incentives for making 

programs look affordable when they are not; (2) facilitates better decisions about 

which programs to pursue and which not to pursue given existing and expected 

funding; and (3) balances the near-term needs of the joint warfighter with the longterm need to modernize the force. Achieving this affordable portfolio will require 

strong leadership and accountability. Establishing a single point of accountability 

could help the department align competing needs with available resources. 

The department has tough decisions to make about its weapon systems and portfolio, and stakeholders, including military Services, industry, and Congress, have to 

play a constructive role in the process toward change. Reform will not be achieved 

until DOD changes its acquisition environment and the incentives that drive the behavior of its decisionmakers, the military Services, program managers, and the defense industry. 

Mr. Chairman, this concludes my prepared statement. I would be happy to answer 

any questions you may have at this time. 

CONTACTS AND ACKNOWLEDGEMENTS

For further information about this statement, please contact Michael J. Sullivan 

(202) 512–4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. 

Individuals who made key contributions to this statement include John Oppenheim, 

Charlie Shivers, Dayna Foster, Matt Lea, Susan Neill, Ron Schwenn, and Bruce 

Thomas. 

Chairman LEVIN. Thank you so much, Mr. Sullivan. 

Dr. Gansler? 

STATEMENT OF HON. JACQUES S. GANSLER, CHAIRMAN, DEFENSE SCIENCE BOARD TASK FORCE ON INDUSTRIAL 

STRUCTURE FOR TRANSFORMATION 

Dr. GANSLER. Mr. Chairman, members of the committee, thank 

you very much for this honor of appearing before you at what I 

think is a critical period and on such an important topic. 

I don’t have to tell this committee of the incredible national security challenges that the United States is facing in the 21st century, 

20 

brought on by the rather dramatic world changes that I believe require a new, holistic view of security—DOD, Department of State, 

Department of Homeland Security, Director of National Intelligence, and so forth—and utilizing both hard and soft power, and 

addressing a very broad spectrum of the security missions with 

great unpredictability and covering the full spectrum, from terrorism all the way through nuclear deterrence. 

I would also emphasize that we need to take full advantage of 

globalization of the technology of industry, not restricting or gaming the benefits from globalization through restrictive legislation. 

In recognizing the long-term national security implications of the 

global financial crisis, the need for energy security, worldwide 

pandemics, the impact of climate change, growing antiglobalization backlash, and the challenging U.S. demographics. We 

have to do all of this, as Senator McCain emphasized, in a likely 

fiscally-constrained budget environment. 

Now, to address these challenges, I believe four highly interrelated acquisition issues must be addressed, and they have to be 

addressed by both DOD and Congress. First, what goods and services to buy; that’s the requirements process. Second, how to buy 

them; that’s the acquisition reform. Third, who does the acquiring; 

we have major issues in the acquisition workforce. Fourth, from 

whom it is acquired; namely, the industrial base. 

Now, I wish I could tell you that there is some silver bullet to 

address all of these needed changes, but it truly requires a very 

broad set of initiatives in each of the four areas if the Nation is 

to achieve the required 21st century national security posture. 

In my prepared testimony, which, Mr. Chairman, I would appreciate being put into the record—— 

Chairman LEVIN. It will be. 

Dr. GANSLER.——I listed the required actions in each of these 

four areas, and I’d be pleased to discuss any of these with you at 

any time. However, for now let me summarize. 

I believe this is a very critical period, perhaps somewhat similar 

to the period following the launch of Sputnik or the fall of the Berlin Wall. Today, the security world is changing dramatically, especially since September 11—geopolitically, technologically, threats, 

missions, warfighting, commercially, et cetera—and this holistic 

perspective that I mentioned is required. Moreover, a decade of 

solid budget growth, which I believe will almost certainly change, 

has deferred the difficult choices—for example, between more 20th 

century equipment versus 21st century equipment—and the controlling acquisition policies, practices, laws, and so forth, as well as 

the Services’ budgets and requirements priorities have not been 

transformed sufficiently to match the needs of this new world. In 

fact, there’s still an emphasis on resetting versus modernization. 

Now, leadership is required to achieve the needed changes. You 

look at the literature on culture change, which I think this clearly 

is, two things are required to successfully bring about the needed 

changes. First is the recognition of the need, a crisis. In this case, 

I believe it is a combination of the economic—the budget, if you 

will, crisis—and the changing security needs, along with the shortage of the senior acquisition experienced personnel to address these 

needs. Second, leadership, with a vision, a strategy, and an action 

21 

1Dr. Gansler is Professor and Roger C. Lipitz Chair, as well as Director of the Center for Public Policy and Private Enterprise at the School of Public Policy, University of Maryland. He 

served as Under Secretary of Defense (Acquisition, Technology, and Logistics) from 1997–2001. 

plan. I honestly believe that President Obama, Congress, and Secretary Gates support the needed changes; however, it’s pretty clear 

that the changes can be expected to be severely resisted. Significant change always is. 

I would start, as my highest priority, with the important role of 

the service chiefs and secretaries in recognizing and promoting senior acquisition personnel, military and civilian. Over the last decade, the DOD acquisition workforce has been greatly undervalued. 

DOD leadership now must demonstrate their personal recognition 

of the critical nature of senior experienced acquisition personnel 

and of the smart acquisition practices that they would bring to 

America’s military posture in the 21st century. 

As my second priority, I would emphasize the importance of 

weapons costs as a true military requirement, to achieve adequate 

numbers of weapons in a resource-constrained environment. This 

will require enhanced systems engineering, including cost-performance tradeoffs, throughout both the government and industry, and 

incentives to industry for achieving lower cost. 

By the way, this has been done before; for example, with the 

Joint Direct Attack Munition missile, where the Air Force Chief of 

Staff said it should hit the target and cost under $40,000 each. It 

now sells for under $20,000 and precisely hits the targets. 

Finally, as my third priority, I would emphasize the value of 

rapid acquisition, from both its military and its economic benefits, 

which will require the full use of spiral development, with each 

block based on proven, tested technology and continuous user and 

logistician feedback for the subsequent block improvements, and 

with the option of continuous effective competition, at the prime or 

at the sublevel. If they’re not continuously achieving improved performance at lower costs, then they should be competed. 

Achieving these required changes will take political courage and 

sustained, strong leadership by both the executive and legislative 

branches, working together. I hope, and firmly believe, that it can 

be achieved. The American public, and particularly our fighting 

men and women, deserve it, and the Nation’s future security depends upon it. 

Thank you. 

[The prepared statement of Dr. Gansler follows:] 

PREPARED STATEMENT BY HON. JACQUES S. GANSLER 1

Mr. Chairman, members of the committee, thank you for the honor of appearing 

before you at this critical period, and on such an important topic. 

I need not tell you that the U.S., in the 21st century, faces incredible national 

security challenges—brought on by dramatic world changes that require: 

• A new, Holistic View of Security (e.g., DOD, State, DHS, DNI, etc.)—utilizing 

both ‘‘hard’’ and ‘‘soft’’ power 

• Addressing a Broad Spectrum of Security Missions—with great unpredictability (from Terrorism to Nuclear Deterrence) 

• Taking full advantage of Globalization (of Technology, Industry, etc.) 

• Recognizing the long-term national security implications of: 

• The global financial crisis 

• The need for energy security 

• Worldwide pandemics 

22 

• The impact of climate change 

• The growing anti-globalization backlash 

• The challenging U.S. demographics 

• To do all of this in a likely fiscally-constrained budget environment 

To address these challenges, four, highly-interrelated acquisition issues must be 

addressed (by the DOD and Congress): 

• What goods and services to buy (the ‘‘requirements’’ process) 

• How to buy them (‘‘acquisition reform’’) 

• Who does the acquiring (the acquisition workforce) 

• From whom is it acquired (the industrial base) 

I wish I could tell you that there was a ‘‘silver bullet’’ to address the needed 

changes; but this truly requires a broad set of initiatives in each of the four areas— 

if the Nation is to achieve the required 21st century national security posture. 

This need, for the four sets of broad changes, was emphasized in a recent Defense 

Science Board report; where they found: 

• ‘‘DOD policies, processes, and management of the Defense Acquisition 

Enterprise (broadly defined) impede the transition to an effective, agile, and 

affordable overall, joint military force for the 21st century.’’ 

• ‘‘U.S. Government policies, practices, and processes do not facilitate the 

development, deployment, and support of the innovative, affordable, and 

rapidly acquired weapons, systems, and services needed for the 21st century forces.’’ 

• ‘‘The absence of many of the needed skills, (e.g., experienced program 

management, systems engineering, biotech, advanced IT) in DOD’s acquisition workforce, (particularly at the senior military and civilian levels), combined with the coming retirement and prior, large acquisition workforce reductions, significantly impedes the development, production, support, and 

oversight of the military capabilities needed for the 21st century.’’ 

• ‘‘Government acquisition policies and industry trends (e.g., further horizontal and vertical consolidations) will not produce the required competitive, responsive, efficient, and innovative National Security Industrial 

Base.’’ 

So let me (very briefly) summarize the changes required in each of the four, critical (and interrelated) areas: [in priority order within each category] 

What is acquired: 

To meet the wide range of challenges, within a resource-constrained environment, 

the Nation must focus on: 

1. Lower cost systems and services 

2. Optimized, net-centric systems-of-systems (vs. individual ‘‘platforms’’) 

3. A ‘‘reserve’’ of resources to rapidly respond to combat commanders’ urgent needs 

4. More ‘‘balanced’’ allocation of resources (to address ‘‘irregular’’ operations): C3ISR, unmanned systems, Special Forces, ‘‘Land Warriors,’’ 

cyberdefense, etc; [and these resources must be moved from the 

Supplementals into the base budget] 

5. Interoperability of ‘‘joint’’ systems; and coalition systems 

6. Planning, equipping, and exercising ‘‘as we’ll fight’’: with allies, multiagencies, and ‘‘contractors on the battlefield’’ 

How goods and services are acquired: 

To achieve higher performance at lower costs and faster: 

1. Require ‘‘cost’’ as a design/military ‘‘requirement’’ (because cost, in a resource-constrained environment, is numbers) 

2. Provide viable, continuous ‘‘competition options’’ (as the incentive for 

higher performance at lower costs) e.g. competitive prototypes, competitive 

split-buys, etc. 

3. Fully utilize ‘‘spiral development,’’ with demonstrated technologies (because it is lower cost, lower risk, faster to field; maintains the option of 

competition; avoids obsolescence; can respond rapidly to combat needs) 

4. Make maximum use of commercial products and services (at all levels—utilizing Other Transactions Authority; especially at lower tiers) 

5. Institutionalize a ‘‘Rapid acquisition,’’ parallel process (to respond to 

COCOM urgent needs) 

23 

6. Create incentives for contractors to achieve desired results (in cost, 

schedule, and performance) 

7. Implement modern, integrated, enterprise-wide IT systems (logistics, 

business, personnel, etc.)—including linking Government and Industry 

8. Address Conflict of Interest concerns (from LSI‰Make/Buy‰SETA); 

but don’t reduce the value of relevant experience 

Who does the acquiring: 

A flexible, responsive, efficient, and effective acquisition program (for sophisticated, hi-tech goods and services) requires ‘‘smart buyers.’’ 

This depends on both quantity and quality of senior and experienced military and 

civilian personnel (especially for expeditionary operations). In the last decade-plus, 

this ‘‘requirement’’ has not been met! In fact, the acquisition workforce declined on 

seniority and quantity even as procurement appropriations increased. 

Therefore, one of the Nation’s highest priorities (not just in the DOD) must be 

to address the acquisition workforce. 

• The DOD, especially, has an acquisition workforce problem: 

• Greatly reduced senior officers and SESs 

• In 1990 the Army had five general officers with contract background; 

in 2007 they had zero. 

• In 1995, the Air Force had 40 General Officers in Acquisition, today 

24; and in 1995, 87 SESs and today 49 

• The Defense Contract Management Agency (25,000 people in 1990 

down to 10,000 today; and 4 General Officers to 0) 

These reductions (due to the under valuing of the importance of the acquisition 

workforce) introduce ‘‘opportunities’’ for ‘‘waste, fraud and abuse’’ (e.g., 90 fraud 

cases under review from war zone; examples of poor acquisition process results, such 

as the Air Force Tanker, the Presidential Helicopter, etc.). These Government acquisition workforce issues must be addressed. I believe that President Obama, Congress, and Secretary Gates all agree on this (but it will take the priority attention 

of the Service Chiefs and Secretaries to make it a priority). 

From whom goods and services are acquired: 

To quote, again, from the recent Defense Science Board study (on the desired 21st 

century defense industry): 


303ful2.eps
24 

‘‘The last two decades have seen a consolidation of the Defense Industry 

around 20th Century Needs—The next step is DOD leadership in transforming to a 21st Century National Security Industrial Structure.’’ 

The ‘‘vision’’ for this 21st Century National Security Industrial Base (which appropriate government actions, i.e. acquisitions, policies, practices, and laws, must 

incentivize and facilitate) are: 

1. Efficient, responsive, technologically advanced, highly-competitive (at 

all levels, including public and private sectors) 

2. Globalized (utilizing ‘‘best in class’’)—requires significant changes to 

U.S. export controls (i.e., changes to ITAR, EAR, etc.) 

3. Healthy (profitable); and investing in IR&D and capital equipment 

(rules should separate IR&D and B&P) 

4. Includes commercial firms and equipment, and maximizes dual-use facilities and workforce (barriers must be removed) 

5. ‘‘Independent’’ systems-of-systems architecture and systems engineering firms (to support the Government—as the integrator) 

6. Merger and Acquisition policy guidelines to be based on this vision 

7. Strong Government-Industry Communications encouraged 

8. All non-inherently-governmental work to be done competitively (public 

vs. private, for current government work) 

9. Structural changes to eliminate appearance, or reality, of conflict of interest (regarding ‘‘vertical integration’’)—but great care to assure relevant— 

experienced firms and people involved 

In summary, I believe this is a critical period, perhaps similar to the period following the launch of Sputnik or the fall of the Berlin Wall. Today the security world 

is changing dramatically—especially since September 11, 2001 (geopolitically, technologically, threats, missions, warfighting, commercially, etc.)—and a holistic perspective is required (including State, DHS, and DNI, as well as coalition operations). 

Moreover, a decade of solid budget growth—which will almost certainly change—has 

deferred difficult choices (between more 20th century equipment vs. 21st century 

equipment). The controlling acquisition policies, practices, laws, etc. and the Services’ budgets and ‘‘requirements’’ priorities have not been transformed sufficiently 

to match the needs of this new world (in fact, there is still an emphasis on ‘‘resetting’’ vs. ‘‘modernization’’). 

Leadership is required to achieve the needed changes! All of the literature on ‘‘cultural change’’ (which this clearly must be) state that two things are required to successfully bring about the needed changes: 

• Recognition of the need (a ‘‘crisis’’) 

In this case, I believe it is the combination of economic/budget ‘‘crisis,’’ the changing security needs, and the shortage of the senior acquisition-experienced personnel 

to address the needs; and 

• Leadership—with a ‘‘vision,’’ a ‘‘strategy,’’ and an ‘‘action plan.’’ I believe 

that President Obama, Congress, and Secretary Gates support the needed 

changes. However, the changes can be expected to be severely resisted—significant change always is! 

I would start with the important role of the Service Chiefs and Secretaries in recognizing, and promoting senior acquisition personnel (military and civilian) in order 

to demonstrate their personal recognition of the critical nature of smart acquisition 

practices to American’s military posture in the 21st century. As my second priority, 

I would emphasize the importance of weapons costs as a military requirement (to 

achieve adequate members of weapons in a resource-constrained environment)— 

which will require enhanced systems engineering (throughout both government and 

industry) and incentives to industry for achieving lower cost systems. Finally, as my 

third priority, I would emphasize the value of ‘‘rapid acquisition’’, for both its military and economic benefits—which will require the full use of ‘‘spiral development’’ 

(with each ‘‘block’’ based on proven/tested technology, and continuous user and logistician feedback, for subsequent ‘‘block’’ improvement—and with the option of effective competition (at the prime and/or sub-level, if they are not continuously achieving improved performances at lower and lower costs). 

Achieving these required changes will take political courage and sustained, strong 

leadership—by both the executive and legislative branches (working together). I 

hope, and firmly believe, it can be achieved. The American public, and particularly, 

our fighting men and women, deserve it—and the Nation’s future security depends 

upon it. 

Thank you. 

25 

Chairman LEVIN. Dr. Gansler, thank you so much. 

Dr. Kaminski? 

STATEMENT OF HON. PAUL G. KAMINSKI, CHAIR, COMMITTEE 

ON PRE-MILESTONE A SYSTEMS ENGINEERING, AIR FORCE 

STUDIES BOARD, NATIONAL RESEARCH COUNCIL 

Dr. KAMINSKI. Mr. Chairman, Senator McCain, and members of 

the committee, first of all, I want to thank you for your leadership 

on these critical acquisition issues and for the invitation to testify. 

Since you’ve asked me to testify, first, in my role as Chairman 

of the National Research Council’s Study on Pre-Milestone A Systems Engineering, with your permission I would ask that my statement, which includes a full summary, be put in the record, and 

then I will proceed to provide a short verbal summary of the summary. 

Chairman LEVIN. All the statements will be made part of the 

record. Thank you. 

Dr. KAMINSKI. Thank you, sir. 

Recent years have seen a serious erosion in our ability to field 

new weapon systems quickly in response to changing threats, as 

well as a large increase in the cost of these weapon systems. Our 

programs today for developing weapon systems take two to three 

times longer than they did 30 years ago. I note that time is money 

in this process, and time also leaves room for disruptions, uncertainty, and changes in commercial technology. In a 15- or 20-year 

period, we’re seeing commercial technology turnover three, four, or 

five times. When a weapon system takes 15 to 20 years to develop, 

the technology that you start with isn’t going to be supported when 

it’s fielded. So, we have to vigorously attack this time issue. 

Our committee also noted the importance of systems engineering 

in reducing this acquisition time, when combined with development 

planning. We further underscored the importance of an early systems engineering effort, in that, the decisions made prior to and 

the key Milestone A and B decisions impact somewhere between 75 

percent and 85 percent of the total life-cycle cost. So, the time to 

address those issues is up front, before those decisions are made. 

Our committee also noted that many of the conclusions that we 

reached have been reached in several previous studies. So, the 

issue isn’t disagreement on what the recommendations are, the 

issue is implementing those recommendations. So, once again, we 

thank you for your leadership in creating a forum for that kind of 

implementation. 

Let me address now the issues that you asked me to address. 

First of all, just one overall comment on systems engineering. I 

agree with Secretary Gates, who, when asked about acquisition, 

said, ‘‘There is no one silver bullet that is going to correct all the 

problems.’’ But, I do believe that good systems engineering, coupled 

with effective development planning early on, are two of the most 

important contributors to successful acquisition. 

Our report provided some formal definitions of ‘‘systems engineering,’’ but they tend to be arcane, so I thought I might start 

with a couple of examples. I’ll briefly describe some examples of 

good systems engineering in the work we’ve done, and also where 

have we seen poor systems engineering. 

26 

One of the really good examples is the Apollo program. That program, from a dead start, put man on the moon in about 8 years. 

When that program was started, we didn’t have mature technology. 

What we did was good upfront systems engineering and development planning, so we could proceed in a sequential way, step by 

step, with each new step building on the previous step. In building 

hardware, we were also building the experience of our acquisition 

workforce and our industry, so we could, step by step, increase our 

capabilities, eventually going to the moon. 

Another really good example is the Air Force Intercontinental 

Ballistic Missile (ICBM) programs that were done in the 1970s and 

1980s. What we saw there is that we would never start a full-scale 

development contract for a new ICBM until we had done the upfront systems engineering and development planning. The development planning produced an inertial guidance system for the ICBM, 

as well as critical propulsion components, and a reentry vehicle. 

That not only reduced the risk of the hardware development and 

integration in the future, it also gave domain experience to our key 

people in government and industry, so that when we threw the 

switch and started full-scale development, we could typically expect 

a first flight in 3 to 4 years, as a result of that experience base. 

That’s what we need to restore. 

You asked: ‘‘What were systemic contributors to acquisition problems?’’ I listed five. 

The first of these is the lack of this early and continuing systems 

engineering, coupled to a development planning program early on, 

right upfront. 

The second key impediment is the lack of alignment of responsibility, authority, and accountability of the program manager. A program manager needs to be able to exercise his or her judgment. 

Much of the program manager’s authority has been taken away by 

one-size-fits-all approaches to acquisition and by the oversight process, which has some onerous elements that are nonvalue-added. 

A third major impediment is the lack of stability in program 

funding. Many contribute to that. 

A fourth is the lack of early attention to test and evaluation, 

with insufficient planning and investment in the tools, such as 

modeling and simulation, test equipment, facilities, and personnel, 

to provide us with the timely and meaningful results needed by 

program management and for continuous systems engineering to 

refine our performance objectives and development plans. 

Finally, the root fundamental issue here is this excessive time to 

acquire that I had spoken about. Time is money. As this time increases from a few years in the past to 15 years today, it undermines our entire process, causing the key participants to lose what 

I call the ‘‘recipe’’ for how we move forward and also to lose a sense 

of accountability. When we see new capabilities that are developed 

and fielded in 5 years, the engineers, the managers, the testers, the 

cost analysts are all able to benefit and apply their experience from 

previous programs, and they can also be held accountable, since 

they can be in place managing the programs deliverables during 

one assignment. That all changes when we move to 15-year acquisitions and we have five rollovers of management, engineers, and 

cost analysts, and five rollovers of the technology in the process. 

27 

So, attacking acquisition time is fundamental. I would say, a testament to our failure today is the fact that we have to discard our 

current acquisition approaches to deal with our urgent needs and 

field systems, such as the Mine Resistant Ambush Protected vehicle and counter-improvised explosive devices (IEDs) by forming and 

using rapid-reaction organizations, because our existing ones don’t 

work. They can’t respond to the cycle time that we need. 

So, what do we do about this? Again, I’ve listed in my statement 

five steps. 

The first is to ensure that we not only restore, but enhance, this 

early and continuing systems engineering work, coupled with development planning. This means restoring funding upfront in the 

programs, and using independent estimates to ensure we have 

enough funding upfront. It also means attracting best and brightest to the critical systems engineering work, and providing a path 

to career advancement, career tracking, and leadership for the key 

people that we need to rebuild in cadre. 

Second issue is the alignment of the responsibility, authority, 

and accountability of the program manager. I’ve listed several steps 

in my statement about what’s needed to be done to do that. 

Third issue is improving funding stability. We pay a great deal 

for the instability we cause by making funding adjustments to program. My experience shows that every time we make a cut in a 

program, for financial or other reasons unrelated to performance, 

we end up eventually putting in three times what we cut to restore 

the program later and get it back to a base. 

Fourth item is giving early and serious attention to the test and 

evaluation issues that I noted earlier, so they can be part of a rapid 

process. When we wait for test and evaluation results because we 

haven’t done a good job planning and preparing, what we have is 

hundreds of people sitting on their hands, waiting for results, and 

we’re paying all those people while they wait for results. 

Finally, the last item is fundamentally attacking this problem of 

long development times by the combination of the previous four 

items. 

I believe action on these five issues will have a significant and 

demonstrable impact on our serious acquisition problems. I believe 

that we need to move now with the same urgency and priority that 

we expect in combat operations to permit the timely and effective 

development and fielding of new capabilities and services with 

what I expect will be more limited future defense dollars. 

Thank you, Mr. Chairman. 

[The prepared statement of Dr. Kaminski follows:] 

PREPARED STATEMENT BY DR. PAUL G. KAMINSKI

Chairman Levin, and Ranking Member McCain: 

Thank you for your leadership on the Department of Defense (DOD) acquisition, 

and for the invitation to testify on these important acquisition issues. Since you 

have asked me to testify in my role as Chair, Committee on Pre-Milestone A Systems Engineering, Air Force Studies Board, National Research Council, I will begin 

by providing a summary of our report, which was approved by the Governing Board 

of the National Research Council and published in 2008. The report is available to 

the public at http://www.nap.edu. After the report summary, I will provide my personal views on systems engineering and respond to the key issues you asked that 

I address. 

28 

1This is a result of the elimination in the 1990s of the development planning function that 

had existed in the Air Force Systems Command. 

SUMMARY

Recent years have seen serious erosion in the ability of U.S. forces to field new 

weapons systems quickly in response to changing threats, as well as a large increase 

in the cost of these weapons systems. Today the military’s programs for developing 

weapons systems take two to three times longer to move from program initiation 

to system deployment than they did 30 years ago. This slowdown has occurred during a period in which threats have been changing more rapidly than ever and when 

technology advances and accumulated experience should have been accelerating 

rather than slowing the development process. 

Many causes for this trend have been suggested, including the increased complexity of the tasks and the systems involved from both technological and human/ 

organizational perspectives; funding instability; loss of ‘‘mission urgency’’ after the 

end of the Cold War; bureaucracy, which increases cost and schedule but not value; 

and the need to satisfy the demands of an increasingly diverse user community. The 

difficulty of focusing on a specific, homogeneous, post-Cold War threat made problems even worse. Yet although the suggested causal factors have merit, a common 

view is that better systems engineering (SE) and development planning could help 

shorten the time required for development, making it more like what it was 30 

years ago. 

Simply stated, SE is the translation of a user’s needs into a definition of a system 

and its architecture through an iterative process that results in an effective system 

design. SE applies over the entire program life cycle, from concept development to 

final disposal. 

The Committee on Pre-Milestone A Systems Engineering was tasked by the U.S. 

Air Force to examine the role that SE can play during the defense acquisition life 

cycle in addressing the root causes of program failure, especially during the preMilestone A and early phases of a program. Currently, few formal SE processes are 

applied to Air Force development programs before the Milestone A review.1

The committee devoted considerable time and space in its report to trying to define a minimum set of systems engineering processes. The most important of these 

processes are summarized in the checklist in Box S–1 below. A few of the things 

that need to be taken care of before Milestone A and just after it are the following: 

the consideration of alternative concepts (solutions) up front; the setting of clear, 

comprehensive key performance parameters (KPPs) and system requirements; and 

early attention to interfaces and interface complexity, to the concept of operations, 

and to the system verification approach. It is these early-stage processes that are 

covered in this report. The importance of stable requirements and funding between 

Milestone B and the achievement of initial operational capability (IOC) is stressed, 

as are processes including good configuration management and change control. The 

committee further stresses in the report what it regards as six of the most important process areas in its discussion of six ‘‘seeds of failure’’. 

29 

Figure S–1 DOD life-cycle acquisition process. Points A, B, and C at the top of 

the figure represent Milestones A, B, and C. LCC, life-cycle cost. Source: Richard 

Andrews, 2003, An Overview of Acquisition Logistics. Fort Belvoir, VA; Defense Acquisition University. Available at http://www.afcea.org/events/pastevents/documents/ 

Track4Session4AMCEmphasisonCustomerFocusedITInitiatives.ppt#364,12,Slide 12. 

Last accessed on November 20, 2007. 

SYSTEMS ENGINEERING AND THE DOD ACQUISITION LIFE CYCLE

The use of formal systems engineering practices throughout the life cycle of an 

acquisition program is critical to fielding the required system on time and within 

budget. Across the top of Figure S–1 are the points at which important management 

decisions are made: Milestones A, B, and C. Concept development and refinement 

occur before Milestone A, and further technology development, to reduce system design and development (SDD) risk, occurs before Milestone B. Only after Milestone 

B does a program become an enterprise with dedicated funding. Importantly, Figure 

S–1 shows that about three-quarters of total system life-cycle costs are influenced 

by decisions made before the end of the concept refinement phase at Milestone A, 

while about three-quarters of life-cycle funds are not actually spent until after Milestone C. This means that although high-quality SE is necessary during the entire 

acquisition cycle, the application of SE to decisions made in the pre-Milestone A period is critical to avoiding (or at least minimizing) cost and schedule overruns later 

in a program. Much of the value of early, high-quality SE will be manifested as success in fulfilling Milestone B requirements. 

MAIN FINDINGS AND RECOMMENDATIONS

The committee’s main findings and recommendations are given below. 

Finding 

Attention to a few critical systems engineering processes and functions particularly during preparation for Milestones A and B is essential to ensuring that Air 

Force acquisition programs deliver products on time and on budget. 

Today’s weapons systems provide unprecedented capabilities but also involve complex interfaces with external command, control, and communications systems and 

rely on a greater volume of software than ever before. Early decisions on the weapons system requirements and capabilities have a disproportionately large impact on 

program cost and schedule. The committee also recognizes that a lack of flexibility 

(a result of overly rigid processes or a lack of trust among program participants or 

stakeholders) can limit the ability of a program manager to change early decisions 

that warrant changing. 

The committee found many gaps and inconsistencies in the way that the Air Force 

manages pre-Milestone A activities. The committee heard from presenters of some 


303ful5.eps
30 

cases for which required documents were completed pro forma and filed away, never 

to be seen again, or for which required steps were skipped completely. The current 

practice of initiating programs at Milestone B denies the acquisition review authority the earlier opportunity (at Milestone A) to make judgments about the maturity 

of the technologies on which the program is based and to decide whether technologies need to be further developed prior to making a Milestone B commitment 

to system development and demonstration. 

Recommendation 

The Air Force leadership should require that Milestones A and B be treated as 

critical milestones in every acquisition program and that a checklist such as the 

‘‘Pre-Milestone A/B Checklist’’ suggested by the committee (see Box S–1 in this 

Summary) be used to judge successful completion. 

A rigorous, standard checklist of systems engineering issues should be addressed 

by each program through both the pre-Milestone A and pre-Milestone B phases. The 

committee’s recommended 20-item checklist is shown in Box S–1. 


303ful6.eps
31 


303ful7.eps
32 


303ful8.eps
33 

While the committee considers that each item on the checklist is important, it 

calls attention to several items that warrant further discussion. Item 2 recognizes 

that the world changes too fast to be friendly to long development cycles. The committee believes that the Air Force should strive to structure major development programs so that initial deployment is achieved within, say, 3 to 7 years. Thirty years 

ago, this was a typical accomplishment—for example, nearly 40 years ago, the Apollo program put the first man on the Moon in fewer than 8 years. 

The development time issue is addressable by applying systems engineering to 

Items 3, 4, and 13 through 15 before Milestones A and B. The definition of clear 

KPPs by Milestone A and clear requirements by Milestone B that can remain stable 

through IOC can be essential to an efficient development phase. It is also important 

that critical technologies be sufficiently mature prior to starting SDD. The committee observed that although today’s systems are not necessarily more complex internally than those of 30 years ago, their ‘‘external complexity’’ often is greater, because today’s systems are more likely to try to meet many diverse and sometimes 


303ful9.eps
34 

contradictory requirements from multiple users. This kind of complexity can often 

lead to requirements being changed between Milestone B and IOC, and it can lead 

to relying on immature technology. 

Item 19 of the checklist stresses the importance of placing experienced, domainknowledgeable managers in key program positions. The committee has observed 

that many of the truly extraordinary development programs of the past, such as 

Apollo, the Manhattan Project, the early imaging satellite programs, the U–2, the 

fleet ballistic missile system, and nuclear submarines, were managed by relatively 

small (and often immature) agencies with few established processes and controls. In 

that environment, dedicated managers driven by urgent missions accomplished feats 

that often seem incredible today. 

The committee believes that the accumulation of processes and controls over the 

years—well meant, of course—has stifled domain-based judgment that is necessary 

for timely success. Formal SE processes should be tailored to the application. But 

they cannot replace domain expertise. In connection with item 19, the committee 

recommends that the Air Force place great emphasis on putting seasoned, domainknowledgeable personnel in key positions—particularly the program manager, the 

chief system engineer, and the person in charge of ‘‘requirements’’—and then empower them to tailor standardized processes and procedures as they feel is necessary. 

One key pre-Milestone A task is the analysis of alternatives (AoA), which entails 

evaluating alternative concepts and comparing them in terms of capabilities, costs, 

risks, and so on. Checklist items 1 through 4, 12, and 13 should be completed before 

the AoA, while items 5 through 11 and 14 through 20 may be addressed after the 

AoA. 

Finding 

The creation of a robust systems engineering process is critically dependent on 

having experienced systems engineers with adequate knowledge of the domain relevant to a contemplated program. 

While the systems engineering process has broad use, effective application depends on having domain experts who are aware of what has gone wrong (and right) 

in the past, recognize the potential to repeat the successes under new circumstances, and avoid repeating the errors. 

Ideally, a person or persons with domain knowledge would have had experience 

working on exactly the same problem, or at least a problem related to the one at 

hand. If that is not so (and it might not be if the problem has never been addressed 

before, as was the case for Apollo and nuclear submarines), the term could be taken 

to refer to academic training in the relevant field of engineering or science. It could 

also refer to the practice of critical thinking and problem solving that comes with 

learning to be a systems engineer and then building on that foundation to gain the 

experiential knowledge and understanding of engineering in the context of an entire 

system. Systems engineering is enabled by tools that have been developed to assist 

in the management of systems engineering (not to be confused with the practice of 

systems engineering). 

Both industry and Air Force presenters told the committee that there are not 

enough domain-knowledgeable and experienced systems engineers to support all of 

the programs that need them. 

Recommendation 

The Air Force should assess its needs for officers and civilians in the systems engineering field and evaluate whether either its internal training programs, which 

include assignments on Air Force programs that provide mentoring by experienced 

people and hands-on experience in the application of systems engineering principles, 

or external organizations are able to produce the required quality and quantity of 

systems engineers and systems engineering skills. Based on this assessment, the Air 

Force first should determine how and where students should be trained, in what 

numbers, and at what cost, and then implement a program that meets its needs. 

The Air Force needs to attract, develop, reward, and retain systems engineers 

across the full spectrum of relevant domains, engage them in the early (pre-Milestone A) phase of new programs (or modification programs), and sustain their participation throughout the life of the programs. One important step in this process 

would be to create an Air Force occupational code for systems engineering so that 

engineers’ experience and education can be tracked and managed more effectively. 

The Air Force should support an internal systems engineering career track that rewards the mentoring of junior systems engineering personnel, provides engineers 

with broad systems engineering experience, provides appropriate financial comVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00038 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
35 

pensation to senior systems engineers, and enables an engineering career path into 

program management and operations. 

Finding 

The Government, federally Funded Research and Development Centers (FFRDCs), 

and industry all have important roles to play throughout the acquisition life cycle 

of modern weapons systems. 

Since the need for a new or upgraded weapons system is most often first recognized by the military user, it is appropriate for the military to codify its requirements and, with support from FFRDC and independent systems engineering and 

technical assistance contractors, to explore materiel and nonmateriel solutions (such 

as doctrinal, organizational, or procedural changes) as well as to assess the potential 

for new technology to provide enhanced capabilities. While it is appropriate and 

usually desirable to engage development contractors in the pre-Milestone B process 

using competitive study contracts, the source selection for system development and 

demonstration should not be made until after the work associated with Milestones 

A and B is complete. 

Recommendation 

Decisions made prior to Milestone A should be supported by a rigorous systems 

analysis and systems engineering process involving teams of users, acquirers, and 

industry representatives. 

Working together, government and industry can develop and explore solutions 

using systems engineering methodology to arrive at an optimal systems solution. 

Finding 

The Air Force used to have a development planning organization that applied preMilestone A systems engineering processes to a number of successful programs, but 

that organization was allowed to lapse. 

The role of the Air Force development planning organization, which was within 

the Air Force Systems Command, was to provide standard evaluation tools and perform pre-Milestone A systems engineering functions across acquisition programs. 

The early 1990s saw an erosion of this front-end planning organization along with 

its funding as the Air Force Systems Command (now the Air Force Materiel Command) began to play a decreasing role in program execution. In the opinion of several speakers who met with the committee, one main reason for the erosion of funding was a lack of congressional support for the planning function. 

Recommendation 

A development planning function should be established in the military departments to coordinate the concept development and refinement phase of all acquisition 

programs to ensure that the capabilities required by the country as a whole are considered and that unifying strategies such as network-centric operations and interoperability are addressed. 

The Air Force and the other military services should establish a development 

planning organization like that which existed in the early 1990s. 

The roles and functions of the various organizations involved in acquiring major 

weapons systems need to be clearly defined. The responsibility for executing systems 

engineering and program management in the pre-Milestone A and B phases should 

be vested in the military departments that do the actual development planning 

functions. This should not be the responsibility of the Office of the Secretary of Defense (OSD) or of the Joint Staff. Instead, those offices need to enable the creation 

and functioning of military department development planning organizations with 

policy measures and, where appropriate, resources. The Joint Staff, under the auspices of the Joint Requirements Oversight Council, may help to define the requirements for major programs in the course of the development planning process, but 

it should not run the process itself. 

The existence of ‘‘joint’’ programs or a program such as Missile Defense, which 

has several related systems being developed by different military services, requires 

clear guidance from both OSD and the Joint Staff about who is in charge. These 

programs need to be harmonized and integrated by the responsible integrating 

agency. However, development planning activities should still take place in the military departments where the expertise resides. Consequently, the development planning should be managed by that agency. 

While this committee cannot predict how Congress will view the revival of a good 

planning process to support pre-Milestone A program efforts, it is still important for 

the Air Force and DOD to make the case for the critical importance of this process 

before Congress and others. A development planning process is important not to 

start new programs, but rather to ensure that any new program (or a new start of 

36 

2GAO, 2003, Defense Acquisitions: Improvements Needed in Space Systems Acquisition Management Policy, September. Available at http://www.gao.gov/new.items/d031073.pdf. Last 

accessed April 2, 2007. 3GAO, 2005, Space Acquisitions: Stronger Development Practices and Investment Planning 

Needed to Address Continuing Problems, July. Available at http://www.gao.gov/new.items/ 

d05891t.pdf. Last accessed April 2, 2007. 4 NDIA Systems Engineering Division, 2003, Task Report: Top Five Systems Engineering 

Issues in Defense Industry, January, Arlington, VA: NDIA. 5DSB/Air Force Scientific Advisory Board (AFSAB) Joint Task Force, 2003, Acquisition of National Security Space Programs, May, Washington, DC. 6 Ronald Kadish, Gerald Abbott, Frank Cappuccio, Richard Hawley, Paul Kern, and Donald 

Kozlowski. 2006. Defense Acquisition Performance Assessment. Available at http:// 

www.acq.osd.mil/dapaproject/documents/DAPA-Report-web/DAPA-Report-web-feb21.pdf. Last 

accessed on April 2, 2007. 7The Acquisition—‘‘Big A’’—system is often believed to be a simple construct that efficiently 

integrates three independent processes: requirements, budgeting, and acquisition. ‘‘Little a’’ on 

the other hand, refers to the acquisition process that focuses on ‘‘how to buy’’ in an effort to 

balance cost, schedule, and performance; it does not include requirements and budgeting. 

any kind) is initiated with the foundation needed for success. Funding for this planning function needs to be determined by the military services, including both the 

acquisition communities and those (the warfighters) who generate the operational 

requirements. 

CONCLUDING THOUGHTS

Many of the conclusions reached and recommendations made by the committee 

are similar to those of previous reviews. Most of the past recommendations were 

never implemented, so one of this committee’s most critical thoughts relates to the 

importance of implementation. A sampling of key findings and recommendations 

from previous studies follows: 

• Government Accountability Office (GAO) 2,3

• Separate technology development from systems acquisition. Commit to a 

program only if the technology is sufficiently mature. Set the minimum 

Technology Readiness Level (TRL). 

• Stabilize the requirements early. • Employ systems engineering techniques before committing to product development. 

• Employ evolutionary approaches that pursue incremental increases in capability. 

• Address shortfalls in science, engineering, and program management 

staff. 

• National Defense Industrial Association (NDIA) 4

• Increase SE awareness and recognize SE authority in the program formulation and decision process. 

• Incentivize career SE positions within the government. 

• Defense Science Board (DSB) 5

• Overhaul the requirements process. 

• Stabilize acquisition tours. 

• Establish a robust SE capability. 

• Defense Acquisition Performance Assessment (DAPA) 6

• Strategic technology exploitation is a key U.S. advantage. Opportunities 

need to be identified early. 

• The U.S. economic and security environments have changed—for example, there are fewer prime contractors, smaller production runs, reduced 

plant capacity, fewer programs, and unpredictable threats. 

• The acquisition system must deal with instability of external funding. 

• The DOD management model is based on a lack of trust. Quantity of review has replaced quality. There is no clear line of responsibility, authority, 

or accountability. 

• Oversight is preferred to accountability. 

• Oversight is complex, not process- or program-focused (as it should be). 

• The complexity of the acquisition process increases costs and draws out 

the schedule. 

• Incremental improvement applied solely to the ‘‘little a’’ acquisition process 7 requires all processes to be stable—but they are not. 

The committee notes that successful implementation of these recommendations requires the ‘‘zipper concept’’—making connections at all levels, from the senior leadVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00040 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
37 

ership of the Air Force and DOD down to the working levels within key program 

management offices and supervisory staffs. 

COMMITTEE ON PRE-MILESTONE A SYSTEMS ENGINEERING: A RETROSPECTIVE REVIEW

AND BENEFITS FOR FUTURE AIR FORCE SYSTEMS ACQUISITION

PAUL KAMINSKI, Chair, Technovation, Inc. 

LESTER LYLES, Vice Chair, U.S. Air Force, (Ret.) 

DEV BANERJEE, The Boeing Company 

THOMAS BLAKELY, Lockheed Martin Aeronautics Company 

NATALIE CRAWFORD, RAND Corporation 

STEPHEN CROSS, Georgia Tech Research Institute 

GILBERT DECKER, Independent Consultant 

LLEWELLN DOUGHERTY, Raytheon 

JOHN FARR, Stevens Institute of Technology 

JAMES FREY, Frey Associates 

ROBERT FUHRMAN, Lockheed Martin (Ret.) 

DAVID GORNEY, The Aerospace Corporation 

JOHN GRIFFIN, Griffin Consulting 

WESLEY HARRIS, Massachusetts Institute of Technology 

RONALD KADISH, Booz Allen Hamilton 

ROBERT LATIFF, Science Applications International Corporation 

ALDEN MUNSON, Independent Consultant 

MARK WILSON, Mark Wilson Consulting 

Staff 

MICHAEL A. CLARKE, Board Director 

JAMES C. GARCIA, Study Director 

GREG EYRING, Senior Program Officer 

CARTER W. FORD, Associate Program Officer 

WILLIAM E. CAMPBELL, Senior Program Associate 

LANITA JONES, Program Associate 

Having summarized the findings and recommendations of the Committee on PreMilestone A Systems Engineering, let me now add my personal views on systems 

engineering, and the two additional questions that you asked me to address: 1) the 

systemic issues that have contributed to cost, schedule, and performance problems 

in the acquisition of major weapon systems; and 2) the steps that Congress and the 

Department need to take to improve performance of the Department’s acquisition 

programs. 

SYSTEMS ENGINEERING

I agree with Secretary Gates who said that there is no one silver bullet that will 

correct all of the DOD acquisition problems. But I believe that good systems engineering coupled with effective development planning are the two most important 

contributors to successful acquisition. Our report provided formal definitions of systems engineering and development planning that are somewhat arcane. So rather 

than provide further definition, I find it easier to illustrate by choosing examples 

of good and bad systems engineering and development planning. Examples of good 

work include the Apollo Program and the U.S. Air Force intercontinental ballistic 

missile (ICBM) programs (e.g., Minuteman, MX) in the 1970s–1980s. Apollo succeeded in putting men on the moon in about 8 years. At the start of the program, 

almost all of the key technologies were immature. But good systems engineering and 

development planning were applied to develop a systematic approach, reducing risk 

by taking a series of limited steps, and applying the learning and domain experience 

gained from each step to the subsequent step. The U.S. Air Force ICBM programs 

used a similar approach, beginning with conceptual studies and technology development, and holding initiation of full scale development (FSD) contracts until key 

guidance system, re-entry system and propulsion technologies had been demonstrated. As a result, the time from initiation of FSD until first flight was typically 

3–4 years. An example of poor systems engineering is the SBIRS program, in which 

a lack of domain experience and analysis led to a failure to anticipate the possibility 

of severe radio frequency interference between two key payloads—discovering this 

problem years after program initiation. Inadequate systems engineering and development are therefore the first of five items listed below as systemic contributors to 

acquisition problems. 

38 

SYSTEMIC CONTRIBUTORS TO ACQUISITION PROBLEMS

1. The lack of early and continuing systems engineering and the absence 

of a closely-coupled development planning program are a fundamental contributor, as identified in our report. The root causes include: (a) lack of sufficient personnel (in both government and industry) with adequate education, training and domain experience (this includes personnel in requirements development as well as in acquisition); and (b) lack of sufficient front 

end investment necessary to understand the key tradeoffs in cost/schedule/ 

performance, and to identify and address the key risks in a systematic 

manner. 2. Lack of alignment of responsibility, authority, and accountability of the 

program manager. In many cases the program manager’s authority is diffused by many levels of oversight in both the Department and in Congress, 

and the financial and performance constraints imposed do not allow sufficient freedom of action to apply informed judgment in a timely manner. 

Flexibility is further limited by application of a ‘‘one-size-fits-all’’ approach 

imposed by the DOD 5000 system, and the oversight practiced by the DOD 

and Congress. A program manager needs the freedom to tailor the acquisition approach to the problem, to ensure that the program response time will 

fall within the response time of the threat, and to apply a variety of tools 

and techniques (such as the use of prototypes, competitive prototypes, modeling and simulation, critical subsystem and component demostration). For 

this to work, we need program managers with the education, training, and 

domain experience needed to enable timely responses and excellent judgment relevant to the domain. 

3. Lack of stability in program funding. 

4. Lack of early attention to test and evaluation, with insufficient planning and investment in the tools (e.g., modeling and simulation, test equipment, facilities, and personnel) to provide the timely and meaningful results needed by program management and systems engineering to continually refine performance objectives and development plans. 

5. Excessive (and growing) time from program initiation to fielding. As 

this time increases from a few years to 15 years or more, it undermines the 

entire acquisition process by causing key participants to ‘‘lose the recipe’’, 

and lose a sense of accountability as well as a sense of being able to make 

a difference. When new capabilities are developed and fielded in 5 years, 

engineers, managers, testers, cost analysts, etc. are able to benefit from and 

apply the experience gained from a previous program or program phase. 

They can also see the results of their decisions and be held accountable. We 

can also meaningfully employ past performance of the contractor as a factor 

in the award of future programs—an important factor in incentivizing contractor performance. This all changes dramatically when the time extends 

to 15 years, and we have five roll-overs of management, engineers, cost analysts, and commercial technology during this time period. This long and 

growing time period is a result of the inflexibility inherent in our entire 

system of requirements development, budgeting and acquisition, and it creates a vicious cycle in which it further exacerbates the contributors above, 

and they in turn further increase the time and cost growth. We see the result when we must discard our current acquisition system in order to deal 

with urgent needs and field systems such as MRAP and jammers to counter 

IED’s by forming and using rapid reaction organizations. This cycle must 

be broken by attacking the root causes. 

STEPS THAT CONGRESS AND THE DEPARTMENT NEED TO TAKE

1. The first step is to insure that we not only restore, but enhance early 

and continuing systems engineering coupled with effective development 

planning. This will require commitment of more significant investment dollars earlier in our acquisition programs, and a commitment to build a cadre 

of systems engineers and development planners with the education, training and domain experience needed to be effective. Attracting ‘‘best and 

brightest’’ to this work—and keeping them—will require a personnel system 

that will identify and track these important human resources and establish 

a career path to allow those who are successful to advance to senior program management and leadership positions. Their domain experience will 

be enhanced by managing the building of critical subsystems during the development planning program, reducing risk and building skills and experience at the same time. Congress and the Department can assist by proVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00042 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
39 

viding incentives for attracting and keeping key personnel (not only financial incentives, but educational, training, recognition, and most important— 

the opportunity to take on challenging developments and see that they can 

make a difference). We will need metrics to assess how well we are doing 

in building and applying this cadre, and we must recognize that this will 

not be accomplished in 4, or even 8 years. But we must begin in earnest 

and begin now. Finally, we need a means to insure that we have adequate 

funding upfront for new programs; one approach would require a report at 

program initiation from an independent cost analyst working with system 

engineers and development planners who have developed their skills on 

previous programs. 

2. Alignment of the responsibility, authority, and accountability of the 

program manager requires that a degree of trust be established between 

the program manager and those responsible for our oversight mechanisms. 

We must be prepared to delegate authority to the program manager, and 

provide him or her with some flexibility to manage—to adjust levels and allocation of funding, to adjust the allocation of performance parameters, to 

adjust schedule, and to tailor the acquisition approach to be responsive to 

the need. Clearly, there must be bounds established beyond which the program manager must seek approval from oversight authorities. But I believe 

these bounds are too narrow and inflexible today. One size does not fit all 

programs. Congress and the Department should be willing to consider and 

tailor many of the restrictions which unnecessarily limit and delay program 

managers today. I have seen many of our successful classified special programs benefit from greater management flexibility than that afforded to 

their in conventional program counterparts. The good managers of these 

special programs have used that flexibility to the benefit of the program 

and the Department by operating with transparency and maintaining trust. 

I realize that it seems counterintuitive to recommend greater flexibility and 

trust in an environment rife with acquisition problems, but I believe we 

need to break the current cycle. One way to begin is with a limited number 

of pilot programs, with first priority to those programs addressing urgent 

needs, and assignment of our most experienced program managers to meet 

those urgent needs. Since these programs will be moving with dispatch, 

they offer the best opportunity to produce early indications of whether this 

is a sound approach which should be extended to other programs. 

3. Improving funding stability will require that the Department and Congress be willing to give up some of their flexibility in making annual (or 

more frequent) adjustments in funding. Doing so will require tradeoffs of 

the costs and benefits, and I believe it is time to make explicit consideration 

of these tradeoffs. I have seen the projected benefits of stable funding by 

looking at theoretical Monte-Carlo simulations (which show efficiency improvements of perhaps 8–10 percent as a result of holding a small capital 

Reserve of less than 10 percent). We can also see the benefits of multiyear 

procurements saving similar or greater amounts. I have also seen many examples in which funding cuts of x dollars today result in later additions of 

3x dollars to catch up. 

4. Giving early and serious attention to test and evaluation will require 

strengthening our test and evaluation organizations and personnel. Test 

and evaluation is often an afterthought, and contracts are often written 

without any mention of how we will test the product. We spend large 

amounts of money when a large development team waits for test results. 

The alternative is to spend less money and time by considering testing and 

investment in test resources as part of our systems engineering and development planning efforts. The actions recommended in paragraph 1 above 

are the same actions required to address these critical test and evaluation 

needs. 

5. Reducing the time from program initiation to fielding will require the 

combination of all actions suggested above. Further benefits will be derived 

by placing more emphasis on time-certain acquisition. This will be helped 

by better development planning and alignment of incentives. With good development planning, we can assign managers to develop prototypes, critical 

systems or components needed to better understand cost/performance 

trades and reduce risk. It is reasonable to expect that many of these developments can be completed in 2–4 years, so one manager will be in place 

from start to delivery. This will help align authority and accountability in 

both government and industry. As these critical subsystems are delivered 

and tested, the risk reduction and domain experience gained in both govVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00043 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
40 

ernment and industry will allow us to reduce the time required to develop, 

integrate and test the full system. We can also apply meaningful incentive 

programs to link profits to demonstrated performance, and use that performance as a factor in making future competitive awards. We can rely on 

the experience gained during development planning to apply informed judgment to adjust requirements to improve value, reduce time, and better estimate and manage costs. The Department and Congress can assist by placing more emphasis on time-certain acquisition, with the opportunity for 

milestone reviews at the completion of major development planning activities. 

I believe action on these five issues will have a significant and demonstrable impact on our serious acquisition problems. We need to move now with the same urgency and priority that we expect in combat operations to permit the timely and 

effective development and fielding of new capabilities and services with what I expect will be more limited future defense dollars. 

Chairman LEVIN. Thank you so much, Dr. Kaminski. 

Mr. Adolph? 

STATEMENT OF CHARLES E. (PETE) ADOLPH, CHAIRMAN, DEFENSE SCIENCE BOARD TASK FORCE ON DEVELOPMENTAL 

TEST AND EVALUATION 

Mr. ADOLPH. Chairman Levin, Senator McCain, members of the 

committee, I’d like to thank you for inviting me today. 

I chaired a recent Defense Science Board (DSB) study of developmental test and evaluation, and during my opening remarks, I’ll 

summarize the key points from the study. I ask that my written 

testimony, which addresses the major findings and recommendations in more detail, be put into the record. 

Chairman LEVIN. It will be. 

Mr. ADOLPH. The task force was originally convened in 2007 to 

investigate the causal factors for the high percentage of programs 

completing initial operational test and evaluation in recent years, 

which have been evaluated as not operationally effective and/or 

suitable. 

The task force was asked to assess roles and responsibilities for 

test and evaluation oversight in the Office of the Secretary of Defense (OSD). We were also tasked to recommend changes to facilitate the discovery of suitability problems earlier, and thus, improve 

the likelihood of operational suitability during initial operational 

test and evaluation. 

Very early in the study, it became obvious that the high suitability failure rates were the result of systemic changes that had 

been made to the acquisition process, and that changes in test and 

evaluation alone could not remedy poor program formulation and 

execution. 

A number of major changes in the last 15 years have had a significant impact on the acquisition process. First, congressional direction from 1996 through 1999 reduced the acquisition workforce, 

which, of course, includes developmental test and evaluation. In 

many instances, services acquisition organizations went well beyond the mandated cuts, some making up to 60 percent reductions 

in organizations providing acquisition support. 

Concurrent with acquisition reform, the general practice of reliability growth during development was deemphasized and, in most 

cases, eliminated. This departure from the widely recognized best 

practice may not have been a direct result of acquisition reform, 

41 

but may instead be related to the loss of key personnel and experience, as well as shortsighted attempts to save acquisition funds at 

the expense of increased sustainment and life-cycle costs. Numerous studies have conclusively demonstrated that investing in system reliability during development will yield a substantial reduction in support costs. 

Our study reached the conclusion that the single most important 

step necessary to correct high suitability failure rates is to ensure 

that programs are formulated to execute a viable systems engineering strategy, including a robust reliability, availability, and maintainability program, as an integral part of design and development. 

Moving on to government test organizations, in the last 15 years, 

with some exceptions, there’s been a significant decrease in government involvement in test planning, conduct, and execution. One of 

our task force members observed that, in many instances, the government has gone from oversight to insight to out-of-sight. Our 

task force recommends that government test organizations reconstitute and retain a cadre of experienced test and evaluation personnel to perform the test oversight function. 

Regarding OSD roles and responsibilities for test oversight, the 

study team found that the developmental test office, which had existed for decades, was disestablished in the late 1990s. Currently 

there is no OSD organization with comprehensive developmental 

test oversight, responsibility, authority, or staff. We recommend 

that the office be reestablished as a direct report to the Deputy 

Under Secretary for Acquisition and Technology, as outlined in the 

proposed legislation. 

I’d like to make a few additional observations about the systemic 

issues that have contributed to the current problems. 

First, during a time of increased programmatic and technical 

complexity, there has been a loss of a large number of the most experienced management and technical personnel without an adequate replacement pipeline. Solutions to acquisition problems must 

begin with reconstituting a trained and experienced government acquisition workforce, which includes program managers, subjectmatter experts, as well as systems engineers, contracts personnel, 

testers, and evaluators. 

Second, more attention must be paid to technology readiness, to 

include prototyping and testing crucial technologies. 

Finally, I believe that the major recommendations in the recent 

study chaired by Dr. Kaminski on pre-Milestone A systems engineer would, if implemented and combined with a revitalized acquisition workforce, go a long way towards correcting many of the current acquisition problems. 

Thank you. 

[The prepared statement of Mr. Adolph follows:] 

PREPARED STATEMENT BY CHARLES ‘‘PETE’’ ADOLPH

Mr. Chairman, members of the committee, I am Pete Adolph, the chairman of a 

recent Defense Science Board (DSB) Task Force study of Developmental Test and 

Evaluation (DT&E). I am pleased to present a summary of the study results. The 

findings and recommendations I will discuss represent a consensus of the Task 

Force members and do not reflect an official position of the Department of Defense 

(DOD). 

A DSB Task Force on DT&E was convened in the summer of 2007 to investigate 

the causal factors for the high percentage of programs entering Initial Operational 

42 

Test and Evaluation (IOT&E) in recent years which have been evaluated as both 

not operationally effective and not operationally suitable. The following are the specific issues which the Task Force was asked to assess: 

• Office of the Secretary of Defense (OSD) organization, roles, and responsibilities for Test and Evaluation (T&E) oversight. Compare organization, 

roles, and responsibilities in both DT&E and OT&E. Recommend changes 

that may contribute to improved DT&E oversight, and facilitate integrated 

T&E. 

• Changes required to establish statutory authority for OSD DT&E oversight. Title 10, U.S.C. has an OT&E focus, and does not address OSD authority in oversight of DT&E. Recommend changes to title 10 or other U.S. 

statutes that may improve OSD authority in DT&E oversight. 

• Many IOT&E failures have been due to lack of operational suitability. 

Specific problems have been in the materiel readiness sustainment areas of 

reliability, maintainability, and availability. Recommend improvements in 

DT&E process to discover suitability problems earlier, and thus improve 

likelihood of operational suitability in IOT&E. 

PROBLEM DEFINITION

In recent years, there has been a dramatic increase in the number of systems not 

meeting suitability requirements during IOT&E. Reliability, Availability and Maintainability (RAM) deficiencies comprise the primary shortfall areas. DOD IOT&E results from 2001 to 2006 are summarized in Figures 1 through 3. These charts 

graphically depict the high suitability failure rates during IOT&E resulting from 

RAM deficiencies. 

43 


303ful3.eps
44 

Early in the DSB study, it became obvious that the high suitability failure rates 

were the result of systemic changes that had been made to the acquisition process; 

and that changes in DT&E could not remedy poor program formulation and execution. Accordingly, the Task Force study was expanded to address the broader programmatic issues, as well as the issues previously identified. 

A number of major changes in the last 15 years have had a significant impact 

on the acquisition process. First, congressional direction in National Defense Authorization Acts for Fiscal Year 1996, 1997, 1998, and 1999 reduced the acquisition 

workforce (which includes DT&E). Several changes resulted from the implementation of Acquisition Reform in the late 1990s. The use of commercial specifications 

and standards was encouraged, unless there was justification for the use of military 

specifications. Industry was encouraged to use commercial practices. Numerous military specifications and standards were eliminated in some Service acquisition organizations. The requirement for a reliability growth program during development was 

also deemphasized, and in most cases, eliminated. At the same time, systems became more complex, and systems-of-systems integration became more common. Finally, there was a loss of a large number of the most experienced management and 

technical personnel in government and industry without an adequate replacement 

pipeline because of the personnel cuts. The loss of personnel was compounded in 

many cases by the lack of up-to-date standards and handbooks, which had been allowed to atrophy, or in some cases, eliminated. It should be noted that Acquisition 

Reform included numerous beneficial initiatives. There have been many programs 

involving application of poor judgment in the last 15 years that can be attributed 

to acquisition/test workforce inexperience and funding reductions. It is probable that 

these problems would have occurred independently of most Acquisition Reform initiatives. 

All Service acquisition and test organizations experienced significant personnel 

cuts, the magnitude varying from organization to organization. Over time, in-house 

DOD offices of subject matter experts (who specialized in multiple areas, such as 

promoting the use of proven reliability development methods) were drastically reduced, and in some cases, disestablished. A summary of reductions in developmental 

test personnel follows. The Army essentially eliminated their military Developmental Testing (DT) component and declared the conduct of DT by the government 

to be discretionary in each program. The Navy reduced their DT workforce by 10 

percent but no shift of ‘‘hands-on’’ government DT to industry DT occurred. The 

trend within the Air Force gave DT conduct and control to the contractor. Air Force 

test personnel have been reduced by approximately 15 percent and engineering personnel supporting program offices have been reduced by as much as 60 percent in 

some organizations. The reduction of Acquisition Program Office and Test personnel 


303ful4.eps
45 

in the Services occurred during a time when programs have become increasingly 

complex (e.g., significant increases in software lines of code, off-board sensor data 

integration, and systems of systems testing). 

PRINCIPAL FINDINGS AND RECOMMENDATIONS

Reliability, Availability, and Maintainability 

As a result of industry recommendations in the early 1970s, the Services began 

a concerted effort to implement reliability growth testing as an integral part of the 

development process. This implementation consisted of a reliability growth process 

wherein a system is continually tested from the beginning of development, reliability problems are uncovered, and corrective actions are taken as soon as possible. 

The Services captured this practice in their reliability regulations, and DOD issued 

a new military standard on reliability, which included reliability growth and development testing as a best practice task. The goal of this process from 1980 until the 

mid-1990s was to achieve good reliability by focusing on reliability fundamentals 

during design and manufacturing rather than merely setting numerical requirements and testing for compliance towards the end of development. 

The general practice of reliability growth was discontinued in the mid- to late 

1990s. This discontinuance may not be a direct result of Acquisition Reform, but 

may be related instead to the loss of key personnel and experience, as well as shortsighted attempts to save acquisition funds at the expense of increased life-cycle 

costs. With the current DOD policy, most development contracts do not include a 

robust reliability growth program. The lack of failure prevention during design, and 

the resulting low initial Mean Time Between Failure and low growth potential are 

the most significant reasons that systems are failing to meet their operational reliability requirements. 

According to Army studies, almost 90 percent of the sustainment costs are directly 

correlated with the reliability of the system. Given the amount of resources consumed during sustainment, investments in reliability enhancements can provide a 

very large return on that investment. A case study conducted by the Logistics Management Institute, provided data that indicated an investment in total program reliability would yield a substantial reduction in support costs. 

Findings 

• Acquisition personnel reductions combined with acquisition system changes in 

the last 15 years had a detrimental impact on RAM practices 

• With some exceptions, the practice of reliability growth methodologies 

was discontinued during System Design and Development 

• Relevant military specifications, standards, and other guidance were not 

used 

• Suitability criteria, including RAM, were de-emphasized 

• Improved RAM decreases life-cycle costs and reduces demand on the logistics 

system 

• The Deficiency Report can be a valuable tool for early identification of RAMrelated suitability problems, when used in conjunction with an adequately 

resourced deficiency correction system 

Recommendations 

The single most important step necessary to correct high suitability failure rates 

is to ensure programs are formulated to execute a viable systems engineering strategy from the beginning, including a robust RAM program, as an integral part of design and development. No amount of testing will compensate for deficiencies in RAM 

program formulation. To this end, the following RAM-related actions are required 

as a minimum: 

• Identify and define RAM requirements during the Joint Capabilities Integration Development System, and incorporate them in the Request for Proposal (RFP) as a mandatory contractual requirement 

• During source selection, evaluate the bidders’ approaches to satisfying 

RAM requirements 

• Ensure flow-down of RAM requirements to subcontractors 

• Require development of leading indicators to ensure RAM requirements 

are met 

• Make RAM, to include a robust reliability growth program, a mandatory 

contractual requirement and document progress as part of every major program review 

46 

• Ensure that a credible reliability assessment is conducted during the various stages of the technical review process and that reliability criteria are 

achievable in an operational environment 

• Strengthen program manager accountability for RAM-related achievements 

• Develop a military standard for RAM development and testing that can 

be readily referenced in future DOD contracts 

• Ensure a adequate cadre of experienced RAM personnel are part of the 

Service acquisition and engineering office staffs 

Roles and Responsibilities of Government Test and Evaluation Organizations 

The role of the government in the DT process has evolved over the past 50 years. 

With some exceptions, there has been a significant decrease in government involvement in test planning, conduct and execution, in the last 15 years. 

The traditional role of the government during the DT planning phase included the 

identification of the test resource requirements and government test facilities, the 

development of the test strategy and detailed T&E plans, as well as the actual conduct of T&E. When a program moved from the planning phase to the test execution 

phase, the government traditionally participated in test conduct and analysis; performing an evaluation of the test results for the program office. With some exceptions, this is no longer the case. Until recently, it was recognized that there should 

be some level of government involvement and oversight even when the contractor 

has the primary responsibility regarding planning and execution of the DT program. 

In addition to the reduction in the number of government acquisition and test personnel, the experience level of both government and industry personnel has steadily 

diminished in recent years. A significant percentage of the workforce became eligible 

to retire since 2000, and due to prior downsizing, there has not been a steady pipeline of younger technical personnel to replace them. 

Findings 

The changes in the last 15 years, when aggregated, have had a significant negative impact on DOD’s ability to successfully execute increasingly complex acquisition 

programs. Major contributors include massive workforce reductions in acquisition 

and test personnel, a lack of up-to-date process guidance in some acquisition organizations, acquisition process changes, as well as the high retirement rate of the most 

experienced technical and managerial personnel in government and industry without an adequate replacement pipeline. 

• Major personnel reductions have strained the pool of experienced government test personnel 

• A significant amount of DT is currently performed without a needed degree of government involvement or oversight and in some cases, with limited government access to contractor data 

Recommendations 

• As a minimum, government test organizations should develop and retain a 

cadre of experienced T&E personnel to perform the following functions: 

• Participate in the translation of operational requirements into contract 

specifications, and in the source selection process, including RFP preparation 

• Participate in DT&E planning including Test and Evaluation Master 

Plan (TEMP) preparation and approval 

• Participate in technical review processes 

• Participate in test conduct, data analysis, and evaluation and reporting; 

with emphasis on analysis and reporting 

• Utilize red teams, where appropriate, to compensate for shortages in skilled, 

experienced T&E domain and process experts 

• Develop programs to attract and retain government personnel in T&E career 

fields so that the government can properly perform its role as a contract administrator and as a ‘‘smart buyer’’ 

Integrated Test and Evaluation 

Integrated testing is not a new concept within the DOD, but its importance in recent years has been highlighted, due in part to the growth of asymmetric threats 

and the adoption of net-centric warfare. A December 2007 OSD Test and Evaluation 

Policy Revisions memorandum reinforces the need for integrated testing. Implementation of integrated test concepts has been allowed to evolve on an ad-hoc basis. The 

time has come to pursue more consistency in integrated test planning and execution. 

47 

Collaboration between developmental and operational testers to build a robust integrated test program will increase the amount of operationally relevant data that 

can be used by both communities. DT and Operational Test (OT) planning is separate and this inhibits efforts by the Services to streamline test schedules, thereby 

increasing the acquisition timeline and program test costs. 

Additionally, there is a widely held assumption by many in the OT community 

that only data from independent OT is acceptable for operational evaluation purposes. While not all information from DT may be useable by the Operational Test 

Agency to support IOT&E, a significant amount of developmental test data can be 

used to partially satisfy OT requirements. More importantly, an operational perspective earlier in the developmental process has often proven to be a catalyst to 

early identification and correction of problems. 

DOD policy should mandate integrated test planning and execution on all programs to the extent possible. To accomplish this, programs must establish a team 

made up of all relevant organizations (including contractors, developmental and 

OT&E communities) to create and manage the approach to incorporate integrated 

testing into the T&E Strategy and the TEMP. 

Findings 

• Service acquisition programs are incorporating integrated testing to a 

limited degree through varying approaches 

• Additional emphasis on integrated testing will result in greater T&E 

process efficiency and program cost reductions 

Recommendations 

• Implement OSD and Service policy mandating integrated DT&E/OT&E planning and execution throughout the program 

• Require sharing and access to all appropriate system-level and selected 

component-level test and model data by government DT and OT organizations, as well as the prime contractor, where appropriate 

• Integrate test events, where practical, to satisfy OT and DT requirements 

Operational Test Readiness Review 

Each Service has an Operational Test Readiness Review (OTRR) process. Although it varies from Service to Service, the process generally results in in-depth 

reviews of readiness to undergo an IOT&E event. 

Findings 

• A DOD Instruction requires that ‘‘the Service Acquisition Executive shall 

evaluate and determine materiel system readiness for IOT&E’’ 

• Decision authority is frequently delegated to the appropriate Program Executive Officer 

• Materiel developer is also required to furnish DT&E report to the Under 

Secretary of Defense for Acquisition, Technology and Logistics 

(USD[AT&L]) and Director, OT&E 

• Shortcomings in system performance, suitability, and RAM are usually identified during the OTRR 

• In most cases, the operational test readiness certifying authority is well 

aware of the risk of not meeting OT criteria when major shortcomings exist 

• Because of funding constraints, the low priority given to sustainment, as well 

as the urgency in recent years to get new capabilities to the Warfighter, major 

suitability shortcomings have rarely delayed the commencement of dedicated 

IOT&E 

Recommendations 

• Conduct periodic operational assessments to evaluate progress and the 

potential for achieving pre-determined entrance criteria for operational test 

events 

• Conduct an independent Assessment of Operational Test Readiness prior 

to the OTRR 

• Include a detailed RAM template in preparation for the OTRR • Require the Command Acquisition Executive to submit a report to OSD 

that provides the rationale for the readiness decision 

OSD Test and Evaluation Organization 

The Task Force was asked to assess OSD roles and responsibilities for T&E oversight. T&E has been a visible part of OSD since the early 1970s, reporting to the 

Research and Engineering command section when it was in charge of acquisition 

oversight and subsequently to the Under Secretary of Defense for Acquisition (now 

48 

AT&L). The early T&E office was responsible for all T&E, ranges, resources oversight, and policy. In 1983, Congress established an independent Director, OT&E organization, reporting directly to the Secretary of Defense (SECDEF), responsible for 

OT&E policy, budget review, and assessments of operational effectiveness and suitability. The Live Fire Test (LFT) oversight function was created and added to the 

DT&E office responsibilities in the mid-1980s. Later, the LFT oversight function 

was moved to the DOT&E organization. 

In 1999, the DT&E organization was disestablished. Many functions were moved 

to DOT&E, including test ranges and resources, and joint T&E oversight. Some of 

the remaining T&E personnel billets were eliminated to comply with a congressionally mandated (AT&L) acquisition staff reduction. The residual DT&E policy and 

oversight functions were separated and moved lower in the AT&L organization. 

A 2000 DSB Task Force Study on Test and Evaluation Capabilities recommended 

that DOD create a T&E resource enterprise within the office of the DOT&E to provide more centralized management of T&E facilities. This recommendation ultimately led to removing the test ranges and resources oversight from DOT&E, abandoning the notion of centralized management, and the establishment of the Test Resource Management Center (TRMC) in AT&L (as directed by the National Defense 

Authorization Act for Fiscal Year 2003). 

Findings 

Current policy as of December 2007 mandates that developmental and operational 

test activities be integrated and seamless throughout the system life cycle. There 

must be enough experts in OSD with the ability to understand and articulate lessons learned in early testing and the ability to execute the new T&E policy. That 

policy is to ‘‘take into account all available and relevant data and information from 

contractors and government sources’’ in order to ‘‘maximize the efficiency of the 

T&E process and effectively integrate developmental and operational T&E.’’ 

• Currently there is not an OSD organization with comprehensive DT oversight 

responsibility, authority or staff to coordinate with the operational test office 

• The historic DT organization has been broken up and residual DT functions were moved lower in organization in 1999, and lower yet in 2002 

• Programmatic DT oversight is limited by staff size and often performed 

by generalists vice T&E experts 

• Recruitment of senior field test personnel is hampered by DT’s organizational status 

• Existing residual organizations are fragmented and lack clout to provide 

DT guidance 

• System performance information and DT lessons learned across DOD has 

been lost 

• DT is not viewed as a key element in AT&L system acquisition oversight 

• Documentation of DT results by OSD is minimal 

• Access to models, data, and analysis results is restricted by current practice 

in acquisition contracting, and the lack of expertise in the DT organization 

• TRMC has minimal input to program-specific questions or interaction with 

oversight organizations on specific programs 

• Organizational separation is an impediment 

Recommendations 

• Implementation of integrated and seamless DT and OT will require, at a minimum, greater coordination and cooperation between all testing organizations 

• Consolidate DT-related functions in AT&L to help reestablish a focused, integrated, and robust organization 

• Program oversight and policy, and Foreign Comparative Test (FCT) 

• Have Director, DT&E directly report to Deputy Under Secretary of Defense, Acquisition and Technology (DUSD[A&T]) 

• Restore TEMP approval authority to Director, DT&E 

• Integrate TRMC activities early into DT program planning 

• Make TRMC responsible for reviewing the resources portion of the TEMP 

• If such an organization is established and proves itself effective, consider as 

part of a future consolidation moving LFT back to its original DT location (this 

would require congressional action and DOT&E concurrence) 

Most of the organizational changes recommended above are within the purview 

of AT&L. The LFT change requires the concurrence of DOT&E and a legislative 

change to Title 10 because of the change in reporting official. All the other recommendations can be implemented within current DOD authority. 

49 

Chairman LEVIN. Thank you so much. 

I think we’ll try a 7-minute round. I’m not sure there will be 

time for a second round; I think we have a vote scheduled around 

11:30 a.m., if I’m not mistaken. 

Let me start with this question. Mr. Sullivan, you commented on 

the reform bill, the Weapon Systems Acquisition Reform Act of 

2009, which I’ve introduced with Senator McCain. Each of you, in 

a way, has commented on it. Let me start with you, Dr. Gansler. 

Would you comment on the bill, any parts you like, any parts you 

don’t like, any additions you might be able to recommend at this 

time? 

Dr. GANSLER. Overall, Mr. Chairman, I thought it was in the 

right direction and important. I think that we have to recognize 

that simply writing a memo, passing a law, doesn’t change the system. Each of the areas that you highlighted, I think are in the 

right direction. What we have to be careful of is not going too far 

in one direction. For example, in the conflict of interest, clearly we 

have to avoid conflict of interest, but we don’t have to go so far that 

we have only people who have no experience in those positions. 

That’s the danger of going too far, in terms of the legislation of it. 

In each of the areas, I think that there’s some clarity that could 

be added, but, in general, I think you’ve gone in the right direction. 

For example, you emphasize systems engineering. I think you need 

to define that as including costs so that there’s no ambiguity as to 

whether that’s a pure engineering problem or a cost issue, from a 

design perspective. 

I think the importance of test and evaluation that you highlight 

is clearly something that’s very important. It could be emphasized 

without really changing the title of the office. But, nonetheless, 

when I was Under Secretary, I felt it was a very critical piece. But, 

it’s important, when you talk about test and evaluation, that it be 

viewed as a part of a development process. We learn from that testing. We do it early, as Mr. Adolph indicated, but we also don’t view 

it as a pass-fail final exam because we’re doing spiral development, 

and we’re continuing to learn from the testing as it goes along, and 

some people have tended to think of the test process as a final 

exam. 

In the area of independent cost analysis, it is absolutely essential. On the other hand, we have the Cost Analysis Improvement 

Group (CAIG) office, which I used all the time, and I felt it was 

critically important. The problem is, people don’t want to use their 

numbers. If they say it’s realistically going to cost more, and government and industry want to put in the low bid in order to get 

a program into the overall budget, that’s a management question; 

it’s not a matter of what organization you set up. But, it’s a very 

important function, as you highlighted. 

In terms of getting the combatant commanders involved in the 

requirements process, that was exactly what we intended with the 

Packard Commission and Goldwater-Nichols intended with the establishment of the Vice Chairman of the Joint Chiefs. That was the 

purpose of that statement. Somehow that hasn’t gotten the intended strength, it has been much more the suppliers than the 

users, if you will, the warfighters. I think it is important to get the 

50 

combatant commanders much more into the loop on the requirements process, and I think you’ve gotten that properly emphasized. 

I think, in the acquisition area I would emphasize that projected 

unit costs are a military requirement because that’s one of the 

things we’ve lost in some of the programs. Global Hawk started out 

that way, got off track. You mentioned the JSF. That started off 

as a unit cost being one of the requirements for globalization of 

that program. It got out off track because it lost sight of that unit 

cost as being one of its principal considerations in design. 

As I mentioned, in terms of competition, I think it’s really important that we view competition as an option throughout the programs, at the prime and at the lower tiers, but not as a law. You 

don’t compete for its own sake; you compete when the current contractors, prime or sub, aren’t getting better performance at lower 

cost. That’s their incentive for doing it. If you tell them we’re going 

to compete it anyhow, they have no incentive. If you tell them that 

if they can get higher performance and lower cost, then, in fact, 

they will try to achieve that so that they don’t have to compete in 

the next round, and it’s doing exactly what the government wants 

them to do. 

In the same way, if they achieve the objectives, we ought to use, 

in effect, the same way that the commercial world does: through 

price elasticity. If you get a lower cost, we’ll buy more of them. We 

don’t take the money away and put it in the general treasury, so 

there will be a need to create incentives for industry, as well as 

government, for doing a better job of achieving higher performance 

at lower cost, continuously. 

In terms of conflict of interest, I think we need to focus on some 

structural ways to address conflict of interest. We do this, in terms 

of foreign ownership, through limited liability corporations, in effect, the special boards are set up. Maybe there’s some ways we can 

do that in order to address conflict of interest without the sort of 

blanket requirement that someone in an engineering job shouldn’t 

know anything about that job. That’s wrong. 

Those are the suggestions that I would have. But, overall, I think 

you are definitely going in the right direction with the bill. 

Chairman LEVIN. Thank you. We would welcome any specific 

language changes you would recommend. 

Now, Mr. Sullivan, you already commented. Do you have any additional comments? 

Mr. SULLIVAN. Yes, just briefly. I would say that, given the package that we’ve looked at, we support everything in it. We thought 

it was very well-targeted to the key problems on acquisition programs, and that the idea to give more authority to the combatant 

commanders, in terms of getting urgent needs met, was a good provision. 

I think the most important thing on an acquisition program is, 

at the outset—we’ve all discussed it here—to have more knowledge 

about the requirements that you’re going to build to, before you 

begin, than they have now. 

So, the two provisions that we think are most important—first 

is the cost-estimating provision. We wrote that we don’t see any 

reason why the CAIG actually couldn’t fulfill that position. The important thing there is to probably provide the CAIG with more reVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00054 Fmt 6633 Sfmt 6602 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
51 

sources so they can do their jobs on a more regular basis and rather than just periodically, at every milestone. The other critical 

thing there is that if the director was moved up, out of the bureaucracy a little bit more, the estimates that the CAIG make 

might be looked at as less personality-driven, if you will, depending 

upon who’s in office. 

As Under Secretary of Defense, Dr. Gansler used the CAIG, others may not use it as much. If they’re reporting to a higher level, 

and they owe Congress a report, I think that’ll really improve their 

authority and their visibility and their independence. But, the key 

is always going to go back to the requirement-setting process, and 

jelling that with cost estimates. If you begin with not enough information about what it is you’re going to, what you want to build, 

you are not going to get a good cost estimate. So, it really has to 

be based on knowledge. 

So, the cost-estimating provision, I think, is very good. The systems-engineering provisions that you have in there go a long way 

to providing the knowledge that the cost estimators would need up 

front. 

Chairman LEVIN. Thank you. 

Dr. Kaminski, do you have any comment? 

Dr. KAMINSKI. Yes, sir, I do. I made some specific comments in 

my statement, but let me make a couple of big-picture comments, 

if I may. 

If there’s a direction in which I would try to move language in 

the proposed legislation, it would be to focus, not only on process, 

on oversight mechanisms, but a focus on people; people that make 

this system work. I don’t care how good a process you put in place, 

if you don’t have people who are experienced and know what 

they’re doing, I think you’re going to end up with problems. 

When I say ‘‘people,’’ I am referring very broadly to our acquisition workforce. Think about this for a minute. These people and requirements development skills are also key for those who are doing 

development of requirements. They have to participate in this 

tradeoff process to consider the cost of what they’re asking for and 

how those tradeoff with the performance capabilities that are desired. 

Testing is a critical piece of this process. If you think about it— 

I know my experience is, there isn’t any program I’ve ever worked 

on in which I didn’t know a heck of a lot more about the program 

6 months or a year into it than I did when I started. So this is a 

continuing learning process as we find out things from testing— 

what’s hard to do, what’s easier to do. We need to have a continuing dialogue in this requirements tradeoff loop. 

So, requirements developers and acquisition personnel have to 

have training and experience in these systems engineering tools 

and techniques. We would never let a fighter pilot get into an aircraft without a very extensive training program to prepare him or 

her for that operation. It’s one of the reasons we do so well with 

our forces. When I compare our requirements for operational requirements with the training and education requirements of people 

going into acquisition requirements generation, they pale in comparison. We have to be able to develop the training, education, and 

the domain experience that go with this to make it work. 

52 

I also believe it is worth it for us to look very hard at: How can 

we systematically reduce this long development time? That’s just 

killing us. There are programs in which we can do time-certain acquisition. Time is money, here. By doing good development planning to be able to reduce the risk, following the example of Apollo, 

following the example of our ICBM programs, I believe we can compress that time. But, we need some targets, and some incentives 

to do that. 

We also need to make better and more extensive use of prototypes, in a sensible way. Competitive prototypes for some programs, perhaps single-thread programs for others. We also need to 

realize that one size does not fit all, here. For example, in dealing 

with IEDs, we find that we need to have an acquisition system 

whose cycle time is measured in weeks. That’s a different acquisition system than you need for a strategic bomber. The acquisition 

system has to fit the cycle of the threat that we’re dealing with, 

so we have to tailor it in that way. 

The last comment I would make is that, with respect to implementation, what better place could we find to start to implement 

some of these processes, procedures, and people development than 

in these urgent programs that are doing rapid acquisition. Why 

start there? One, it’s urgent. Two, we’ll be able to see the impact 

of changes that we make more quickly in programs that are operating in cycle times of weeks or months, and see what’s benefiting 

us and what is not. I’d like to see that commitment to implementation because it’ll happen much more quickly than if we simply 

write new processes or new 5000 series in the DOD. 

Chairman LEVIN. Thank you. 

My time’s way up, but, Mr. Adolph, do you have a brief comment 

on the bill? Any changes? 

Mr. ADOLPH. Very brief comment. In general, I believe that the 

proposed legislation will go a long way toward alleviating the major 

acquisition problems which have occurred in recent years. 

I also have a quick comment regarding the legislation pertaining 

to the director of the developmental test function. I would add one 

responsibility, that the director participate in the acquisition program reviews conducted by the Under Secretary for Acquisition 

and submit a status of the developmental testing for the programs 

under review. That was the norm in the past, for many years, and 

I think that’s important. 

Chairman LEVIN. Thank you so much. 

Senator McCain. 

Senator MCCAIN. Thank you, Mr. Chairman. Again, thanks, to 

the witnesses. 

I think there’s general agreement on the part of the witnesses 

that there’s been a dramatic erosion in acquisition workforce and 

test and evaluation personnel. Maybe I could ask you, Dr. 

Kaminski, what happened? 

Dr. KAMINSKI. I think a number of things happened. Just as 

there’s no one silver bullet, there’s no single entity or action to 

blame. But, let me list some of the things that happened. I think, 

as we looked at major programs of acquisition reform, we tried to 

do more with less. We also had some pretty strong direction from 

some portions of Congress on this. I can recall the chairman of a 

53 

key committee who publicly made statements that said we had too 

many ‘‘shoppers’’ in the Department, and we really needed to reduce this acquisition workforce. So, over a period of 3 or 4 years, 

nearly 50 percent of the acquisition workforce was taken out. Now, 

the Department was a partner in that, agreeing to those reductions. 

Senator MCCAIN. We removed incentives for people to remain in 

the acquisition workforce, in the form of lack of promotion or career 

enhancement? 

Dr. KAMINSKI. Exactly. If we are able to attract and train the 

right system engineers—if that system engineer doesn’t see some 

path for advancement in the DOD, he or she is going to go find a 

place where they can make a difference and have an opportunity 

for advancement. 

Senator MCCAIN. Which means a revolving door evolves? 

Dr. KAMINSKI. Yes, sir. But, that revolving door actually may be 

the commercial industry. Again, I return to the comment I was 

making about the importance of cycle time in the acquisition system. If you have someone who’s really worth their salt and able to 

make contributions, and they get into a DOD acquisition system 

which is going to produce something in 15 or 20 years, it won’t take 

them long to realize that their knowledge base is going to erode so 

that they will no longer be valuable to commercial industry, which 

is producing things in 2- or 3-year timeframes. 

So, to be able to have some revolving door from commercial industry back to DOD will benefit from getting these cycle times 

down. 

Senator MCCAIN. Dr. Gansler? 

Dr. GANSLER. Yes, Senator McCain. In my prepared testimony, 

I actually showed a graph that came out of the commission that I 

ran. I was shocked at how much the acquisition workforce has been 

allowed to deteriorate. In fact, as we came out of the Cold War, the 

procurement budgets dropped, the number of acquisition people 

came out with it. Then, as Dr. Kaminski said, there was a legislative mandate in 1996 to take another 25 percent out. So we’ve 

ended up, now, dramatically, where we had about 500,000 people 

in 1990, we now have about 200,000 people. But, the acquisition 

dollars have gone up dramatically, so you have this huge gap between the dollars and the people. 

But, much more important is the point that you just made about 

the officers and the senior people. In 1990, the Army had five general officers; in 2007, when we did the study, there were zero general officers with a contracting background. In the contract management organization, the Defense Contract Management Agency 

(DCMA), they had 25,000 people in 1990, they have 10,000 today. 

There’s basically an undervaluing. They used to have four general 

officers; they have zero. So, as you suggest, if you’re a young major, 

you’re not getting into that career field, and, as a result, it’s been 

just totally undervalued. Without those experienced senior people, 

both civilian and military, they don’t know what questions to ask, 

and they are not going to be able to make the right judgments. 

Senator MCCAIN. Obviously, we need to have some personnel policy changes, as well. Mr. Sullivan, the issue of Nunn-McCurdy— 

when it was first passed, we thought it was really important and 

54 

effective. For a while, it was. I think a breach of Nunn-McCurdy 

was a big deal. Now it seems to be a routine kind of event that the 

notification comes over, we see it, and ho-hum. Are we in danger 

of experiencing the same thing with this measure? 

Mr. SULLIVAN. We have noticed that the Nunn-McCurdy 

breaches we see take a lot longer to resolve and come out with a 

new program, and the funding is continuing on that program as 

they do that. 

As I read the proposal that the committee has now, I think it’s 

really good to have a termination criteria like that. I think GAO 

thinks that’s a good thing. In other words, a program termination 

would not just have someone do a review, but also probably look 

at what happens, what the triggers are for that. I think, right now, 

it states that the program cannot change the scope of work, cannot 

start new contracts, and there might be one other thing that the 

program can do. I don’t know if legislation can basically say that 

that program can no longer obligate money, but that would, to me, 

be a much more direct way to get the point across. If a program 

is automatically terminated when it passes a threshold, and cannot 

obligate money, that might get people’s attention. 

Senator MCCAIN. I can imagine the blowback when some vital 

program is shut down because of our failure to act, but I think 

we’re in agreement that there has to be a more robust oversight 

and ability to exercise that oversight as we see these costs spiral 

completely out of control. 

Mr. SULLIVAN. Yes, sir. 

Senator MCCAIN. Mr. Adolph, on the issue of Nunn-McCurdy, do 

you think it’s sufficient measure to impose more discipline on the 

cost overruns? 

Mr. ADOLPH. I believe it’s sufficient. But, the issue that concerns 

me, and the remedy has already been discussed, is to get the technology readiness right, at the outset; and second, to get a realistic 

cost estimate. There’s too much concurrency, there are actions that 

are taken in the program to kick the can down the road on NunnMcCurdy, which, in the long run, in my opinion, adversely impact 

the program. The example from the F–35 is getting rid of two of 

the test articles. 

Again, I believe Nunn-McCurdy is fundamentally sound. The 

problems in recent years are a combination of issues which I believe the proposed legislation, if it’s really implemented in the Services—and, again, back to Dr. Kaminski and Dr. Gansler’s point— 

in order to implement it, we have to reconstitute the acquisition 

workforce. That’s the first step. 

Senator MCCAIN. Also I think it would be important for us to 

have the combatant commanders more involved in the requirements process, as well. I think sometimes we have neglected that 

aspect of the equation. 

I thank you, Mr. Chairman. I thank the witnesses. 

Chairman LEVIN. Thank you, Senator McCain. 

Senator Akaka. 

Senator AKAKA. Thank you very much, Mr. Chairman. 

I want to add my welcome to the witnesses before us today, and 

to point out that I’m glad that, Mr. Chairman, you are holding this 

hearing on acquisition. For me over the years, I’m beginning to feel 

55 

that we need to change, or try to change, the heart and soul of 

DOD, and, really, the culture of DOD, and to get to acquisition. 

Gentlemen, as you well know, the reform of DOD acquisition 

process is an extraordinarily complex undertaking involving many, 

many moving parts. However, I believe the first step to tackling 

any problem is to prioritize. So, let me ask this question to each 

of you. In order to most effectively reform the DOD acquisition 

process, what do we need to focus on first? 

First, Dr. Gansler. 

Dr. GANSLER. I believe it’s the people. I think if we’ve undervalued the importance of this area, in terms of promotion, in terms 

of experience, in terms of numbers, all across the board, both civilians and military, that we’re not going to get there, even if we pass 

all the laws in the world. We need the people who are going to be 

driving this process. That is my number-one priority. We have neglected it and, of course, in the last 8 years we’ve been living in 

a rich man’s world, so money doesn’t matter, and if people overrun 

or they don’t perform, ‘‘Let’s spend more money.’’ Now, that’s not 

going to be the case, and we need people who are smart, experienced, and competent to run their programs—with flexibility, 

though. They have to make tradeoffs of cost and performance, systems engineering kinds of work, test and evaluation, so forth. That 

requires management judgment, and you can’t just legislate that, 

and therefore, you need people with experience to be able to make 

those management judgments. That’s my number one priority. 

Senator AKAKA. Dr. Gansler, would you also say that another 

part of that would be inadequate staffing? 

Dr. GANSLER. Yes. 

Senator AKAKA. From what I gathered, there were positions that 

were not filled. 

Dr. GANSLER. Absolutely. I was shocked to find that, in Iraq and 

Afghanistan, only 35 percent of the people that were in their jobs 

were qualified for those jobs, even with the minimal qualifications 

that Dr. Kaminski mentioned. Besides that, most of the positions 

weren’t even filled, and they were almost all volunteer civilians in 

the warzone. We need to be able to get some senior military there, 

as well. So, there’s a great lack of people, numbers, but you don’t 

want just numbers, you also want qualified people. Numbers won’t 

do it. It has to be qualified, experienced people. Some of those by 

the way, can come from industry. You can rotate people from industry, without conflict of interest, very easily. That’s what we’ve 

had with many of the people with past experience. We do that in 

DARPA, we bring them in and out, and we’ve done it in other parts 

of the government. I think that people are out there with experience; we just have to make sure that we make it attractive to get 

them in these government acquisition positions. 

Senator AKAKA. Mr. Sullivan. 

Mr. SULLIVAN. I think, at a strategic level, one of the biggest 

problems the DOD faces is that it is unable to prioritize what 

weapon system capabilities should be put into programs. They have 

95 weapon system acquisition programs, major acquisitions, that 

are underway right now. That’s up from about 75 or so in 2000. 

There is a tendency to have too many programs vying for the acquisition dollars that are available to DOD on an annual basis. 

56 

When that happens, you get a very unhealthy competition, where 

many programs and few dollars drive the requirements-setting 

process, which is stovepiped in many ways by the Services. 

So, the Services are all vying for a solution, they want requirements that are very, very tough to make, so that their weapon system can do the most. They, as a result of that, tend to put in optimistic cost estimates. The funders, of course, are looking at those 

very optimistic cost estimates that are very heavy on assumption 

and very light on facts, data, and actual costs. The acquisition process begins with the lack of the systems engineering that we’ve 

talked about here today. 

So, you have too many programs chasing too few dollars, with 

business cases that are unexecutable. The whole system is segregated in such a way with process owners and stakeholders that, 

in a way, it works for everyone. That’s the culture. It’s a performance-driven culture, and we all understand that and accept that, 

but there’s also a lot of players involved in the culture that create 

this kind of unhealthy competition at the outset. I think that’s the 

culture change that has to take place in this DOD, and that is almost intractable, when you think about it: the difficulty of changing that. I think legislation can go a long way to requiring people 

to do certain things, but, in the end, I agree with Dr. Gansler, it 

falls on the people to have the right principles, if you will, to 

change what this system is really supposed to create. 

Senator AKAKA. Dr. Kaminski, with your research and study 

background, do you care to comment? 

Dr. KAMINSKI. Yes. Let me not repeat because I agree with everything that’s been said. Let me put another dimension on the 

people. I, too, would answer: most important is people. But, what 

you have to do is recognize the dynamics that involve people. If 

we’re going to attract our very best and brightest people to an activity, I’ve found that the principal incentive usually isn’t money. 

With the salaries we pay military officers and civilians in DOD, we 

are still able to attract very capable people to key jobs. 

What attracts them? What usually attracts them is the ability to 

make a difference, to see that they can have a personal impact on 

a major program, on the security of this Nation. 

So, I want to come back again to my issue about time. When programs are taking 15 or 20 years, many of those best and brightest 

people say, ‘‘What’s the difference? If I’m not going to see something happen for 15 or 20 years, why don’t I go someplace where 

I can make something happen sooner?’’ If they don’t have any freedom to make decisions and influence things because of excessive 

oversight processes and complexity of the process, they’ll go find 

another place to work. 

I want to share with you a perspective that I got from a different 

position. Rather than a perspective of a previous Under Secretary 

of Defense, I want to share the perspective that I gained when I 

served on Active Duty in the Air Force. I’d say I spent two-thirds 

of my career there working on special access programs, part of that 

career, in the early days of the National Reconnaissance Office, 

where I was a program manager for one of our National Reconnaissance spacecraft that’s up and flying today, and for several years 

in the stealth program. Let me just pick one example there, the

57 

F–117 program. I was heavily involved in that program. When we 

initiated that program, our plan was, from the beginning of fullscale development to field the aircraft in 3 years. We missed that, 

sir. We missed it by a year. We fielded the aircraft in 4 years. Everybody who worked on that program could see that they were 

making a difference. They could see that capability coming along 

to be fielded. There was excellent interation and tradeoffs in that 

process, between the testers and the program managers and the 

users who were going to use that aircraft. We made continual adjustments. So, that motivation of people and reduced time go together. To attract the right kind of people, we have to work on this 

time issue. 

Senator AKAKA. Mr. Adolph? 

Mr. ADOLPH. As I mentioned in my earlier remarks, people are 

certainly the highest priority, and that’s been reiterated here by 

the previous comments. 

Another area for improvement is certainly overly ambitious requirements. We need to continue to push technology, without a 

doubt. But, at the same time, we need to ensure that the technology is sufficiently matured to incorporate in a weapon system. 

That means prototyping and testing, and testing the prototype item 

in an environment in which it’s going to be placed in a combat environment. So, again, first, people. Second, getting the requirements right and making sure that they’re not overly ambitious. 

Again, Dr. Kaminski’s study really addresses the latter issue quite 

well. 

Senator AKAKA. Thank you very much, Mr. Chairman. 

Chairman LEVIN. Thank you, Senator Akaka. 

Senator Collins. 

Senator COLLINS. Thank you. Thank you, Mr. Chairman. 

Mr. Chairman, I first want to salute you and Senator McCain for 

your initiative. I think you’ve brought forth a bill that will really 

make a difference. I also want to point out that I’m very pleased 

to hear the witnesses today all stress the importance of the acquisition workforce. This is an area that Senator Lieberman, Senator 

Akaka, and many of us, have said is the number-one problem, over 

and over again. When I brought this up at the White House Conference last week, however, some of my colleagues felt that it was 

a lesser problem. So, I was very pleased to see the panel of experts 

before us list this as perhaps the greatest problem that we’re facing. 

Dr. Gansler, you and I also worked together, many, many years 

ago, on how to increase competition in Federal contracting, and it’s 

very good to see you here again, as well. 

I want to ask the panel about some of the specific provisions in 

the Levin-McCain bill. In particular, this bill would require that 

costs be considered right up front, when the requirements are set. 

That is a pretty dramatic change from how military requirements 

are set now, when they’re done in an environment that does not 

consider costs, but, rather, an idealized world, where costs would 

not be a factor. 

Dr. Gansler, you endorsed including cost as a design military requirement right up front, so I’m going to skip over you for this 

question and go to the rest of the panel and ask all of you: Should 

58 

costs be considered up front when military requirements are first 

established? 

Mr. Sullivan? 

Mr. SULLIVAN. I think that is a good idea. Dr. Gansler said that 

should be one of the key performance parameters on any major 

weapon systems, in their business case, and I think we would agree 

with that. 

When you’re trying to set requirements, and you’re doing the requirements analysis that’s needed, I think you begin, at first, in an 

unconstrained manner, and try to get from the user what the user 

would like, in an unconstrained environment. But then it’s critical, 

at some point, to start bonding that with the realities of the time 

it’s going to take to get that to the user, the amount of money, and 

the technologies you have available to do it. So, I don’t think you 

could do that without precluding the ability to think, unconstrained. But the exit criteria would be something that is constrained, at least in a cost range. Then once it exits the requirements process, there is a stage before it would become a set business case and begin as an acquisition program, where that cost 

range could be further reduced to more of a point estimate by continuing to make trades. 

Senator COLLINS. Dr. Kaminski? 

Dr. KAMINSKI. Senator Collins, I also agree that cost should be 

an upfront factor. I’d add another factor to go with it, though. 

Senator COLLINS. Yes? 

Dr. KAMINSKI. Back to my comment about time. Time and money 

go together. When we’re doing requirements tradeoffs, if we’re 

going to have an acquisition cycle time that operates within the 

threat cycle time, those developing requirements have to look at 

the time they want something fielded, as well. So, that needs to be 

an important consideration in this process, and we need to manage 

the time. 

The one other comment I wanted to make with respect to cost 

estimates: we talked earlier about breaches in Nunn-McCurdy. The 

surveillance system that finds the breaches, I think, is fine. But, 

one of the things we want to do is look at root causes. What’s causing us to get into Nunn-McCurdy? One of the constructive uses for 

the independent cost estimate that was in the bill might be to add 

another consideration. We’ve talked about the importance of the 

upfront work in systems engineering and development planning. 

One of the things that would be useful for us all to ensure is that 

there is adequate funding at the beginning of a program, between 

Milestone A and B, for us to apply our systems engineering and development planning capabilities to get a good handle on what those 

cost estimates are, and do a thorough job involving stakeholders, 

the requirements part of the equation, and the program manager 

in that process, along with the CAIG. 

I, like Dr. Gansler, used the CAIG very heavily. In fact, I advocated that we fund programs to the CAIG estimate, but we hold 

the program manager and the contractor to the estimates they developed, so we had some finance reserve between the two estimates. 

Senator COLLINS. Thank you. 

Mr. Adolph? 

59 

Mr. ADOLPH. Senator Collins, certainly we need to consider the 

cost issue upfront. Another very important driver early on is technology readiness. In program after program after program, we’re 

into full-scale development and discover that some of the critical 

technologies simply aren’t mature enough. So the program is delayed, and that drives the schedule, and these slippages occur. 

A key to getting the costs right is to ensure that the technology 

is really sufficiently mature. 

Senator COLLINS. Thank you. 

Dr. Gansler, an issue that hasn’t been discussed today is the impact of a lack of stability and predictability on a program’s cost 

growth. That obviously can drive up the unit cost. When the military starts out with one plan for acquiring a weapon system, and 

then switches direction or reduces the number of units, doesn’t that 

also drive up the cost? 

The reason I bring this up is, we talk a lot about the errors made 

by contractors, we talk about weapon systems that get gold-plated 

because additional requirements are added, but there’s also an important issue, as far as the stability of funding and the lack of predictability driving up the unit cost. Could you address that issue? 

Dr. GANSLER. Yes. But, let me just briefly comment on your first 

question, though, because frequently the military doesn’t think that 

cost is a military requirement. What they neglect is the fact that 

numbers are a military requirement. If you’re resource-constrained, 

the total dollars that you have divided by the unit cost gives you 

the numbers. Numbers really matter in military operations, whether it’s by Lancaster’s law of N-squared or by numbers. Either way, 

numbers really matter, and therefore, cost really matters. That’s 

why it’s so important to have the unit cost as part of the requirements. 

Now you get to your changes, and unless we estimate the cost 

of those changes and their impact on the ultimate cost of the equipment, we let things get out of hand. So, if cost is a requirement, 

then every time a change comes in, and as Senator McCain said 

earlier, it was 75 a week on the LCS, you have to price those each 

out to make sure it is not having a big cost impact on the program. 

Then, when programs, in general, become relatively stable, you 

don’t get this ripple effect through the budget, which is the point 

that you’re really making, Senator. If I want to pay for program A, 

I take it out of program B, not recognizing that program B now is 

in really bad shape because they don’t have the stability of the 

funding. That stability of funding is a critical issue. 

Senator COLLINS. Thank you. 

Chairman LEVIN. Thank you, Senator Collins. 

Senator Burris. 

Senator BURRIS. Thank you, Mr. Chairman. 

This is an interesting experience for me, to hear the distinguished testimony, Mr. Chairman. I would rather listen than to 

talk. I yield my time. 

Chairman LEVIN. Thank you. 

Senator Burr is next on this side; he’s not there. 

Senator Chambliss. 

Senator CHAMBLISS. Thank you, Mr. Chairman. 

60 

Gentlemen, it’s good to see all of you, and thanks for your service 

over the years, and thanks for your being here on what I think is, 

if not the most important issue at the Pentagon today, certainly it 

ranks up pretty high. With these budget times that we’re in, trying 

to figure out a way to buy the weapon systems that we need within 

the timeframe we need is virtually impossible. That’s why your testimony is so critical. 

I want to go back to what Senator Collins was talking about 

there, and what all of you have alluded to in some part, and that’s 

this issue of instability, whether it’s requirements, whether it’s personnel, whether it’s funding, or whether, as you say, Mr. Adolph, 

it’s the technology aspect of it, where we, too many times, tend to 

come up with a great idea, and by the time we get into the production phase, we’ve wasted, not only time, we’ve wasted money, but 

oftentimes, there’s a new idea that has been developed in the 

meantime. 

I’m amazed, Dr. Kaminski, that you say, on the F–117, that we 

were in production in 4 years. Gosh, if we had done that with the 

F–22, we wouldn’t be having the arguments we’re having today, 

and we’d have a great airplane, and we’d be worrying about other 

issues. 

But, it doesn’t make any difference whether you’re talking about 

an aircraft carrier, whether you’re talking about FCS or a tactical 

fighter, we come up with this idea, and we get into the R&D phase, 

and there is, all of a sudden, a great idea, but instability in all four 

of those areas runs that cost up tremendously. Then, you throw in 

what Senator Collins alluded to, about the number of these units 

that we’re going to buy, and all of a sudden it explodes again and 

it becomes such a negative at the Pentagon, rather than the positive that it started out to be. 

My question for a comment from each of you is: How do we get 

back to this? How do we get back to the point to where we come 

up with this idea? If it’s a tactical fighter, it’s supposed to be airto-air, or supposed to be air-to-ground, whatever it may be, how do 

we develop that and get it into production right away, without 

technology intervening and all of a sudden having to add this and 

add that? How do we get our arms around that issue? 

Jack, let’s start with you. 

Dr. GANSLER. I think one of the main opportunities we have is 

to accept the concept of spiral development, that for the block 1 

system, we have a fixed set of requirements, we have a fixed price 

that we’re trying to get a fixed schedule, as Dr. Kaminski said, and 

we go ahead with block 1 under the assumption that if we can then 

demonstrate new technology, if we find that the user needs something different, if we find that even the logistician has a problem 

with maintenance of that equipment or the reliability as we deploy 

it, that becomes block 2, block 3, block 4. But, block 1 has to use 

proven technology and get out there quickly, all with a set of constraints. So, it’s a stable program, as you point out. 

The most successful acquisition that, in fact, Congressman Aspin 

used to always highlight, was the Navy’s Polaris, Poseidon, Trident. When I was in industry, I always knew how much money I 

was going to get next year for that program. I could hire, I could 

plan my workforce, and so forth. That stability is very important 

61 

for efficiency. I think, if we go to a spiral development model, 

whereby block 1 is stable, and block 2 is being developed while 

block 1 is being deployed, you have the concept of stability built in, 

and evolutionary systems are still stable. That’s the way the real 

world, the commercial world, works. You constantly are upgrading 

the software, the hardware of computers, but you’re constantly getting higher performance at lower cost. That has to the be objective 

of each of the blocks as we’re going along in spiral development. 

Senator CHAMBLISS. Mike? 

Mr. SULLIVAN. Goldwater-Nichols legislation, from years ago, 

tried to bring jointness into the DOD. I think it succeeded on the 

operations side. We now have the combatant command’s matrix, 

the military forces that fight wars jointly very well. The same thing 

did not occur on the acquisition side. I think if you look at what’s 

going on now, there’s a kind of a stovepipe system for how you get 

programs started, and that creates this kind of competition for big 

requirements and cost estimates that are heavy on assumption. 

I would agree with everything that Dr. Gansler just said. If you 

can work on that and get a more joint requirement setting and 

funding system, and try to get the proper balance of weapon systems started, try to get rid of the stovepipes, you’d have an environment that could do what Dr. Gansler, I think, is describing, a 

little more easily. 

Another program I would throw out is the F–16 program. Back 

in the 1970s, it was a block program. It was the capabilities that 

the Air Force wanted for the F–16. They knew that they needed an 

aircraft faster than they could develop the technologies to get those 

capabilities, so they had blocks. If you look at the F–16s, and, for 

that matter, the F–15s, performance over the last 30 years, it’s 

pretty impressive. They basically upgraded those aircraft pretty efficiently as they went because they started without that big revolutionary leap, that one-step, big-bang kind of a thing. 

So, we’ve done this before, and I think it’s possible to get back 

to it, but this is where the culture comes in. I think there’s some 

culture change that needs to take place. 

Dr. Gansler has other things that I read in his report that would 

help this significantly. Open systems, for example, on these weapon 

systems, when, if you can make interfaces on the weapon systems 

uniform, you can keep proprietary data that subcontractors have 

that supply subsystems to them, and all they have to do is have 

the proper interface. Then you can open up competition. 

One more point I would make is the difference between technology development and product development probably needs to be 

better understood. Technology is the kind of thing you should think 

of when you think of scientists and lab coats and trial and error, 

and it’s done in a smaller-dollar environment, where you can test, 

use trial and error, and make mistakes. You have to keep that off 

of these acquisition programs. I think someone up here, I don’t 

know if it was Dr. Kaminski or Mr. Adolph, said that when you 

have a technology that’s not mature, and it’s on an acquisition program that’s driving towards production, you have an entire workforce—an entire supply chain, for that matter—that’s waiting for 

that technology to mature. The burn rate is pretty big on that 

workforce that you have. 

62 

Dr. KAMINSKI. I’d emphasize that point. While everybody’s waiting, we’re paying, and so what you want to do is decouple those 

two. 

In terms of approaches to do this, I agree completely with the 

spiral development approach. One problem I see is in the application of spiral. In the few spirals I’ve seen that we actually implemented, we had everything but the kitchen sink in the first spiral, 

rather than stretching this out over a period of time, like we did 

in Apollo. So, we need development planning. You have to have a 

plan with stability to do this. The program manager has to have 

the discipline and the experience to reject things that aren’t in the 

plan or that aren’t mature enough to be harvested. 

I have found that one of the characteristics of a good program 

manager is a big lower right-hand drawer in their desk, and what 

went into that drawer were all the ideas for improvement. The 

drawer was kept closed until such time as we fielded the first system; that is the time open the drawer and look to see what development plans we need to deal with shortfalls or upgrades for that 

system. 

This time-certain development is important because, as time goes 

by, the technology gets old, and new ideas are introduced that end 

up being disruptive to the process. So, time is a key factor here. 

This stability issue is really key. If I look back through my whole 

career, there is only one program I ever worked on where we actually produced the system at the rate we planned. That was the F– 

117. We built one a month. Every other program I can think of, by 

the time we were done with development, we couldn’t afford the 

build rate that we planned. 

Senator CHAMBLISS. Anything to add, Mr. Adolph? 

Mr. ADOLPH. Just one thing, since I certainly agree with the 

other panel members, but with my background in tests, I’ll add 

that issue. In the case of the F–15 and F–16, which were mentioned earlier, I worked on both programs, and I was out in the 

field, test business, working for the Air Force. 

Chairman LEVIN. Please talk a little bit louder, if you would. 

Mr. ADOLPH. Yes. In the case of the F–15 and F–16, I was working for the Air Force in the field at the time, and the F–15 test program was structured well; we had 19 test articles and sufficient articles in the case of the F–16 as well. In the latter F–16 case, the 

propulsion system had been matured. In order to move a program 

along, you need an adequate number of test articles, and you need 

to be able to do as much testing as you can. Avionics is a good example, where considerable testing can be accomplished on a test 

bed platform, rather than the developmental platform. 

So, that’s, I think, one key component of keeping a program moving. When a program stagnates, when you only have one or two 

test articles, and you have a ‘standing army’ of test personnel waiting, and the fixed cost of those people is almost as great as the incremental cost of doing additional testing, particularly when you 

recover the article. That’s not the case in a missile program, where 

the test article is destroyed on each test. 

Chairman LEVIN. Thank you, Senator Chambliss. 

Senator McCaskill. 

Senator MCCASKILL. Thank you, Mr. Chairman. 

63 

Let me start with Mr. Sullivan, and there have been several references that sometimes we have excessive oversight with no value 

added. That’s a frustrating comment to me, in that I have watched 

the Defense Contracting Audit Agency basically get taken to task 

by their peers for failure to even follow basic auditing standards. 

I’ve read, I can’t tell how many of your reports that have, in fact, 

identified weapons acquisition as high risk, since 1990. It is not as 

though the oversight’s not occurring, it’s just that it’s being ignored. It’s not adding value because nobody’s paying any attention 

to it. 

Let me ask you, in that regard, about JCIDS, Planning, Programming, Budgeting, and Execution (PPBE), and Future Years 

Defense Program (FYDP). Now, first of all, I think we ought to 

pass a law that they quit talking in initials because you all know 

what those things are I just said, but I guarantee you, nobody that 

I work for in Missouri has any idea what JCIDS is, PPBE is, or 

FYDP. What they are is: one, is all the Services getting together 

and basically giving each other what they want; two, is a 2-year 

calendar-driven process, where they’re supposed to be figuring how 

they’re going to spend the money; and three, it is the Secretary of 

Defense trying to low-ball what it’s going to cost, long-term, in 

order to make sure that the other two go along with it. Is that an 

accurate summary of what those three are? 

Mr. SULLIVAN. The—that—— [Laughter.] 

I would say, number one, that the JCIDS is a requirement-setting process; it’s where all of the Services tend to get together and 

figure out what it is that they require. 

Senator MCCASKILL. But, in your testimony, Mr. Sullivan, you 

pointed out, they never say no. 

Mr. SULLIVAN. That’s true. 

Senator MCCASKILL. Have they ever said no that you’re aware of 

since you’ve been looking at this? 

Mr. SULLIVAN. We issued a report for this committee, I think, 

about a year ago, where we looked at that, and it was—I think the 

JCIDS process then, if I’m not mistaken—this may not be exact, 

but I can get it for you—about 90 percent of the proposals that 

went in were granted. 

[Excerpt of GAO–08–1060 follows:] 

64 


303ful10.eps
65 

Senator MCCASKILL. Explain to me, so I understand, what is the 

makeup of the three groups of people. These three are really the 

stovepipes that cause a lot of the problem because you have, 

‘‘Here’s what we want, here’s how we pay for it, and here’s how we 

figure out how much it’s going to cost in the long run.’’ 

Mr. SULLIVAN. Yes, ma’am. 

Senator MCCASKILL. What I don’t understand is: what is keeping 

them from making that one thing, so they all have to do all of that 

at once? 


303ful11.eps
66 

Mr. SULLIVAN. The requirement-setting process, the JCIDS, is 

run by the JROC, and the JROC is made up of the military service 

chiefs, more or less. It’s run by the Vice Chief of Staff. So, it, in 

a sense, is a matrix organization, but it receives most of the proposals for needs to be validated as weapon systems from the three 

Services, as stovepipes. 

Now, the JCIDS was established to have something called ‘‘functional capability boards,’’ which were supposed to be a matrixed organization based on looking at things like battlespace awareness or 

force protection or force projection, looking at it functionally instead of across the Services. What we found when we did that 

study was that the DOD has not staffed those functional capability 

boards properly, it hasn’t resourced them properly, so they don’t 

really do a lot of joint decisionmaking to send proposals forward to 

the JROC. So, mostly what they are receiving is proposals for capabilities that are coming from the Air Force, the Navy, and the 

Army, and they compete with each other. 

Senator MCCASKILL. The sense is, at JCIDS, that if you say no 

to what they want, then they’re going to say no to what you want. 

Isn’t that part of the problem, in terms of the way this is actually 

supposed to be oversight with no value added. 

Mr. SULLIVAN. I would say that the oversight is far from perfect. 

There’s not a lot of value to it. 

Senator MCCASKILL. The idea of the Joint Staff is that they’re 

supposed to be picking winners and losers. What is your sense of 

how effective the organization of the Joint Chiefs office has been, 

in terms of weapon acquisition and picking winners and losers? 

Mr. SULLIVAN. In terms of that organization that does that picking of the winners and losers, the JROC, there are a lot of 

redundancies right now in the weapon system portfolio because 

they can’t make proper decisions, it seems to us. 

I’ll give you an example. Right now, there are some unmanned 

aerial systems that are in development that we believe, and I think 

the Department actually believes, should have been joint programs, 

but the Services had unique-enough requirements and missions to 

be separate. Right now, there is the Predator, which has been very 

valuable in Iraq and Afghanistan and is an Air Force program. Air 

Force is currently making a bigger, more powerful Predator, called 

the Reaper. Another one of the Services, I believe it’s the Army, 

has started what they call the Sky Warrior. Both drones are done 

by the same contractor and both with very similar requirements, 

but the Services have determined that they’re different enough that 

they each have to have their own acquisition program. That’s the 

sort of thing that the JROC is contending with; it’s a very parochial 

kind of an attitude. 

Senator MCCASKILL. So, is this not fixable? 

Mr. SULLIVAN. This is the cultural aspects of this, I think, that 

we’ve all been addressing, where you can write legislation, you can 

have policies. 

Senator MCCASKILL. It doesn’t do any good. 

Mr. SULLIVAN. But, unless you change the culture, and I guess 

that is the number one question: How do you change that culture, 

that has been in existence for so long, to try to turn it a little bit, 

to do things a little more efficiently? It’s very much a culture issue. 

67 

Senator MCCASKILL. I also think, Mr. Chairman, one of the 

things we have to do, in some instances, is take out a mirror because I think there are times that, when the military has tried, either by the way they’ve done the budget or by actually being so 

bold as to say we need to wind down a program, that Congress decides, because of our parochial interests, that it’s important that we 

go to bat to augment the budget to take care of the weapon system 

that we think is important in our part of the world. So we contribute to this problem, and I think we shouldn’t complete this 

hearing without at least acknowledging that sometimes Congress 

has their hand in this stew. 

Mr. SULLIVAN. Yes, ma’am. I would say, sometimes we should 

think of this as a system that is in equilibrium because there are 

very many stakeholders in this system that are getting specific 

things, even the GAO. It’s a pretty good employment program for 

us. We report on cost schedule and performance problems, and we 

have been doing it 30 years. So, culturally speaking, if you examine 

it as a system that, maybe, is in equilibrium, in a sense—it’s not 

necessarily broken for the people that are involved in it. 

Senator MCCASKILL. Thank you. 

Thank you, Mr. Chairman. 

Chairman LEVIN. Thank you very much. 

Just one quick comment on that. We do, in this bill, attempt to 

make the JROC system cover some of the issues that Senator 

McCaskill talked about by requiring it to make these early tradeoffs, by looking at cost and at schedule, by the way, as well as the 

requirements and the performance requirements. 

Mr. SULLIVAN. Yes, sir. 

Chairman LEVIN. There is an effort in this bill, Senator 

McCaskill. I just want to give you assurance that at least this bill 

attempts to do what we can, legislatively, to put those elements 

into the JROC process, which would, hopefully, cut down the parochialism by forcing consideration of cost and schedule, not just requirements. So, I’d just get that on the record. 

Thank you, Senator McCaskill. 

Senator Martinez is next. 

Senator MARTINEZ. Mr. Chairman, thank you very much. 

Dr. Gansler, I wonder if I could ask of you to define for me a little more broadly what you mention as a ‘‘holistic approach’’ to defense needs. 

Dr. GANSLER. Yes. In fact, when I even looked at the situation 

in Iraq and Afghanistan, I was surprised to find that the State Department, United Sates Agency for International Development, and 

DOD were all there, but not integrated, in terms of the buying, the 

contracting, and planning purposes. But, much more at a higher 

level than that, it seems to me that the world of the 21st century 

is going to require us to combine hard and soft power in the kinds 

of operations involving expeditionary operations or insurgent operations that we get involved with around the world. So, we’re going 

to need a much closer tie between, in that case, State and Defense. 

But, I would go further and say, Homeland Security and Defense. 

It shocked me, in fact, that Saddam Hussein didn’t try and pull 

some terrorist actions at the same time as we attacked him. I 

would expect that’s going to happen in the future. So, it is an inteVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00071 Fmt 6633 Sfmt 6602 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
68 

grated, holistic perspective that is the meaning of ‘‘national security’’ in the 21st century, that will involve Homeland Security, that 

will involve State, will involve the intelligence community, will involve the DOD. 

I think that’s basically what National Security Advisor General 

Jim Jones is now trying to do, some of the restructuring that’s taking place at the national security level within the President’s office. 

I think this combination of soft and hard power is going to be required, very clearly. That’s what I was thinking of, in terms of the 

‘‘holistic perspective.’’ But, also the types of threats. Think about it, 

the energy case, the pathogen spreading worldwide, the economic 

crisis that we’re in, these are all national security issues for the 

21st century that we have to start to incorporate into our thinking 

of national security. That’s what I had in mind. 

Senator MARTINEZ. Thank you. I think it is an intriguing future 

that we all are stepping into. I think, by the way, Saddam would 

have if he could have. 

But, the LCS program is one that I’m very much fond of, and one 

that I think is essential to the national security interests of our 

country. I wonder how you believe the Navy got so far off track on 

that particular program? Was it too many requirements being put 

on the platform by the Navy? Was it the length it’s taken to develop it? We now have two hulls being developed. So, to the extent 

that any of you could speak to the LCS program and what you see, 

going forward, it would be helpful to me. 

Dr. GANSLER. I just actually published, I think last week, a DSB 

report which looks specifically at the LCS and the presidential helicopter, a couple of programs of that sort, where the initial concept 

was: get something relatively fast, take something ‘‘off-the-shelf’’ 

that could be used, and addressing Dr. Kaminski’s point about 

rapid acquisition—then maybe block 2, 3, and 4 would add some 

of the additional things. 

What happened on the LCS: the first thing they said was, ‘‘Has 

to go through Sea State 8.’’ Well, that’s like going through a hurricane and it wasn’t initially designed for that. Then the next thing 

they said was, ‘‘Well, it has to have a new Navy sprinkler system.’’ 

The sprinkler that was in the system in those two ships that you 

talked about, it wasn’t going to be adequate, for some reason or 

other. So, each of these special requirements ended up basically 

changing the original block 1 system and introduced the instability, 

cost growth, and schedule impact that we’ve talked about in all 

these other programs. 

Yes, we badly needed the LCS, but is it going to have to be a 

battleship? Does it have to do everything that a battleship does? 

How it’s going to be used by the Navy was resisted, in terms of the 

nontraditional solutions. So, this is the culture change that we’ve 

been talking about, as well. If you kept the cost and the schedule, 

and got a block 1 system out there much faster—I like the idea 

that we did it competitively. I think that was a very important 

step. So, I would encourage that to be done in these earlier demonstration systems. 

Senator MARTINEZ. Dr. Kaminski? 

Dr. KAMINSKI. Yes, sir. I think the concept initially was a good 

concept. What was missing was the upfront systems engineering 

69 

and development planning that I spoke about. I can remember 

back to the time I was serving in the acquisition executive’s job in 

DOD; I was a big proponent of commercial practice, and also in 

buying commercial systems where that made sense. But, if somebody came to me and said we were going to buy a commercial ship 

for this mission, my first question to him was, ‘‘What are we going 

to change in the mission to be able to do it with a commercial 

ship?’’ If the answer is ‘‘nothing,’’ then I have to ask a second question, ‘‘Wait a minute, this commercial ship doesn’t have the kind 

of military requirements you would have for fire-safe cables, for a 

fire sprinkler system, or a whole variety of things. What are you 

going to do about those?’’ I don’t think we started asking those 

questions about the LCS program until we were well into the program, so we missed this upfront set of tradeoffs. Those are tradeoffs that you have to make. They can be made sensibly if you approach them, understand them, look at the costs, the performance, 

and the schedule to make those tradeoffs. I don’t believe we made 

those tradeoffs upfront. That, for me, does not necessarily damn 

the LCS program. There may still be value derived from looking at 

these tradeoffs and now making sensible decisions to go forward. 

I agree with Dr. Gansler about the advantage of having a competitive environment to be able to do that. 

Senator MARTINEZ. Mr. Adolph? 

Mr. ADOLPH. I’m not really familiar with the LCS. I’ve had no 

involvement with the program. 

Senator MARTINEZ. Okay, and you, Mr. Sullivan? I don’t know 

whether you’ve had any experience. 

Mr. SULLIVAN. I don’t have any specific experience with that, but 

we do have a team that looks at all of our Navy ships, and I’m sure 

we’ve had a report on that recently, which I could submit for the 

record. 

Senator MARTINEZ. That would be great. I’d appreciate that. 

Mr. SULLIVAN. Okay. 

[Excerpt of GAO–05–255 follows:] 

70 


303ful12.eps
71 

Senator MARTINEZ. I’m sure going to try to look up your article, 

as well, Dr. Gansler. 

You haven’t written on DGD–1000 and the DGD–51 to date, 

have you? 

Dr. GANSLER. Not lately. 

Senator MARTINEZ. Okay. Do any of you share an opinion on the 

needs of the Navy as it relates to these two programs? 

Mr. SULLIVAN. I would just say that we’re going to come out with 

our annual assessment of programs, and I think both the DGD– 

1000 and LCS are going to be programs that are covered in that, 


303ful13.eps
72 

so you will get our take on them, probably in the next couple of 

weeks. I’ll follow up, as well. 

Senator MARTINEZ. Very good. Thank you. 

[Excerpts of GAO–09–326SP follows:] 


303ful14.eps
73 


303ful15.eps
74 


303ful16.eps
75 


303ful17.eps
76 


303ful18.eps
77 


303ful19.eps
78 


303ful20.eps
79 

Senator MARTINEZ. My time’s up, thank you very much. 

Chairman LEVIN. Thank you, Senator Martinez. 

Senator Begich. 

Senator BEGICH. Thank you. Thank you very much, Mr. Chairman. 

It has been very interesting, listening to all the testimony. I will 

try to ask my questions very quickly, and I may not ask every single one of you to answer because I want to get some specifics here. 


303ful21.eps
80 

Before I talk about the personnel issue, I’m a former mayor; I’ve 

just become a Senator. So, I believe I’m a mayor who just happens 

to be a Senator, so I like some of the comments you’ve made on 

personnel issues that I’ll talk about in a second. 

I’m not familiar with the CAIG. Is that the CAIG? I looked 

through the bill as quickly as I could, and I see an annual report 

to Congress. Do you think it would be helpful for us to require 

more? I believe it’s a combination of oversight; it’s not just internal 

oversight, but this body needs to do more. I think this bill attempts 

to do that. But, do you think there should be a report—and maybe 

there is and I’m just not familiar with it—that comes to this committee on a regular basis, maybe quarterly or twice a year, that 

shows what the CAIG said it would cost before a system is started? 

Mr. Gansler, you made the comment that—I may get these words 

wrong—but it’s almost like they ignore it. It comes out, it says, 

maybe, what it might really cost, but then they kind of push it 

down. I’m familiar with this, as a mayor. We call them HMS, Inc., 

studies, where the contractor wants to build it, as well the person 

inside the system who wants to build it, always seems to have a 

different price. When it’s all done, it’s pretty close to the one that 

HMS, Inc., did. Is that something we should have in this legislation, so we can see, before these systems start kicking off, here is 

what the real, or what another group said it could cost, so we at 

least have some understanding? I don’t know who can answer that. 

Dr. GANSLER. The idea of the independent cost analysis, right upfront, when we’re doing the early systems engineering, is to be able 

to see the impact of the various requirements, and to be able to 

trade requirements, schedule, and cost as part of that early design 

requirements setting. Cost is actually an engineering challenge, 

just as schedule and technical performance are. So, if you, upfront, 

try to say, ‘‘What’s the cost impact of this?’’ and the independent 

cost analysis group will say, ‘‘Historically here’s what it’s been,’’ 

and then people will come back and say, ‘‘But, this time it’s going 

to be different.’’ 

Senator BEGICH. Everything’s different the next time. 

Dr. GANSLER. Right, and so it’s important to keep that in perspective, that the new technology comes along, and so forth. But, 

it’s absolutely essential to get that independent estimate. Now, do 

you want to have Congress legislate what the price should be? I 

don’t think so. 

Senator BEGICH. No, I’m not asking that. But here we are, complaining about all these cost overruns. I agree with all your comments. You can write all the legislation you want, but you do not 

change the culture and remove people who are not doing the job 

they should be doing, and putting people in there who should be 

doing the job, you don’t change anything, we’ll be back here in a 

couple of years. So, I’m not saying, legislate the price, but we become more aware, so we are putting the pressure where it should 

be. Let’s be honest about the pricing, so when we do the budgeting 

authorization and appropriation, we don’t go from 75 to 95, with 

two-thirds probably in the planning and design stage, and we always get the answer, ‘‘We’re this far, we have to do a little bit 

more, a little bit more, a little bit more.’’ So what I’m asking is, 

should we have a more regular reporting period? Because once 

81 

you’re a year into a project, even though some are longer, they’re 

already obligating more money. By the time this system here 

moves, you could be 2 years into having any commentary on it. 

That’s my question. Should it be more regular reporting? 

Dr. GANSLER. I think the thing you’re missing is the fact that, 

in the development of a program, the program changes weekly. 

There’s always technical issues that come up, there’s always schedule problems, there’s personnel problems, and so forth. You don’t 

want Congress to be micromanaging the programs. You do want to 

make sure that the process is a good process. That’s where I think 

the independent cost analysis is a very important thing and that 

your emphasis on it is the right thing to do. But, I don’t think you 

want to get down to the point where you have a weekly report from 

the DOD. 

Senator BEGICH. I didn’t ask that. I’m trying to get to the point. 

Is annual enough? 

Dr. GANSLER. I think that—if you’ve convinced the DOD that you 

care about this, that you are going to be monitoring it, and that’s 

what you’re trying to do with the threshold numbers and the controls on that—I think that you’re giving the message that the DOD 

needs to care about cost. 

Senator BEGICH. Okay. Let me make one comment. I forget who 

said it, about, ‘‘stop the obligation of money at a certain point.’’ I 

know, as a mayor, that’s what you do. You turn the dollars off, and 

suddenly you get a response, and you get people paying more attention. So, I might be a little different than earlier comments that 

were made. I’m a little more direct on that. But, the personnel 

issue, to me, this is the challenge. If you don’t change the culture, 

nothing changes. We’ll be back here, and the numbers will grow, 

as they have over the years. It’s not about adding more people, and 

I think Congress made a huge error by reducing down the amount 

of people. That was a huge mistake. We basically took the people 

who manage our programs out of the equation. 

So, besides putting more people into the system to make sure we 

have more folks out there, do you think—and I’m not sure I want 

to ask this question because I’m not sure you’ll want to answer it— 

but, do you think, within the system that currently exists today, we 

have to change the deck? When I say ‘‘change the deck,’’ change 

personnel, people, and not just add more people and move people 

around so we satisfy their issues. This is a very hard, direct question. I’ve had to do this, as mayor. You might have half a dozen, 

a dozen, or 100 people; it doesn’t really matter—they’re in the 

wrong place. Anyone dare to want to answer that question? 

Mr. SULLIVAN. I would just say, just real quickly, that a lot of 

this is organizational. I think the people that work in the DOD now 

are great people, and really, really capable people, and good public 

servants and everything else. 

Senator BEGICH. Right. 

Mr. SULLIVAN. But, when we talk about culture, I guess, it’s 

more the way things are organized. For example, I think, in our 

written statement, we have that the Under Secretary of Defense for 

Acquisition, Technology, and Logistics probably should have more 

ability to make the final decisions on things, acquisition, than he 

or she has now. One of the things that probably gets in the way 

82 

of that is the fact that that position had an average turnover of 20 

months since it started in 1986. I think that’s part of the problem; 

there’s an accountability issue, people change over too much, 

there’s not a lot of direct communication. The three processes, I 

think, that this legislation does address, that you’re trying to get 

the three big processes that we’ve talked about to communicate 

with each other more and to share in decisionmaking; right now, 

that’s not there. So, I don’t know if the people are good people. The 

structures and the way they’re organized and the way they come 

and go is the big issue. 

Dr. GANSLER. If I might comment that if we implemented what 

Goldwater-Nichols says to do, relative to promotion potential for 

the acquisition community, that you’d get a big step forward there. 

Instead of putting someone into a four-star position who has no acquisition background, but happens to be called an acquisition job, 

that’s where we lose out. Each of the promotion-potential reviews 

and so forth need to really show that we value the acquisition 

workforce, civilians and military, and that’s a critical point, I think, 

in order to keep people coming in. Dr. Kaminski said they’re not 

doing it for the money, they’re doing it so they can have an impact, 

and they need to have promotion potential. 

Senator BEGICH. I know my time’s out, but you’re about to jump 

out of your seat, Mr. Adolph. 

Chairman LEVIN. Can you do it real quickly? 

Mr. ADOLPH. Very quickly. There are three issues: numbers, 

training, and people. Particularly in the Air Force, they need to 

plus up because they drew down their acquisition workforce to a 

greater extent. 

I think the training that Defense Acquisition University provides 

is on the mark, for the most part. They were a part of our study. 

They ground in what we found into their training courses. I believe 

people need to be moved around a bit more, particularly in the civilian workforce, so that they get a variety of experiences rather 

than 1 or 2 years’ experience 10 times. 

Senator BEGICH. Thank you very much, Mr. Chairman. 

Chairman LEVIN. Thank you, Senator Begich. 

Senator Lieberman. 

Senator LIEBERMAN. Thanks, Mr. Chairman. Thanks to the 

panel. It’s probably not been an exciting hearing in content, but it’s 

actually very important, an educational hearing, and there’s a lot 

on the line in it, so I thank the four of you for the accumulated 

experience and wisdom you’ve brought to the table. I thank Senator 

Levin and Senator McCain for their legislation, S. 454, which certainly would take us forward in significant ways. I hope it passes. 

Earlier, Senator Collins referred to the fiscal responsibility summit that President Obama convened last week at the White House, 

and then a group of us on this committee happened to be in a 

breakout session on procurement reform. As Senator Collins indicated, most, though not all, but most, of us focused on the acquisition workforce, and the size of it. I want to talk to you about that 

in the time I have, hopefully, and at least one other subject we 

talked about. 

Dr. Gansler, you had a chart here in your testimony which has 

two lines; one is procurement by DOD, in dollars, from 1990 to 

83 

2006, about as close as we can get to today, and then the other is 

the acquisition workforce. Obviously, the procurement dollars go up 

dramatically as the acquisition workforce goes down. But, beneath 

that, there are some numbers that are quite stunning, I think, and 

I know when I was here, Mr. Sullivan referred to them. The acquisition workforce, in 1990, was actually 500,000 people, and today 

it’s dropped, but it’s still 200,000 people, which is an enormous 

number of people in acquisition. I note in your testimony that you 

focus in on the DCMA and say that it had 25,000 people in 1990, 

down to 10,000 today. Then the other four general officers, and 

down to zero today. Give me some sense of the 200,000. Because 

my first reaction to it is, ‘‘Wow, 200,000 people, isn’t it enough to 

handle acquisitions by the DOD, even though acquisitions are so 

large?’’ 

Dr. GANSLER. A large share of those are in the military depots 

that you have insisted do 50 percent of the maintenance work. A 

depot that has 20,000 people, that adds up pretty fast. To get to 

200,000, you only need 10 of those. 

Senator LIEBERMAN. So, acquisition wouldn’t be what most of 

those do? 

Dr. GANSLER. That’s part of the acquisition workforce because logistics is part of the acquisition. 

Senator LIEBERMAN. Got it. But in the conventional meaning, I 

think that’s important, so I wanted to bring out. 

Dr. GANSLER. Very few people are actually doing contract work, 

program management work, or things like that. As Mr. Adolph 

pointed out, the test and evaluation community is down significantly, but they’re part of that community. So it’s the total encompassing the research labs that the government runs all the way 

through the maintenance and logistics support. 

Senator LIEBERMAN. Is there any way, now or maybe afterward, 

to submit for the record, you could give us a sense of what we 

would normally, in conversation, consider to be the acquisition 

workforce—how many people in the DOD are actually involved in 

acquisition, contracting, et cetera? 

Dr. GANSLER. It’s a small percentage of the people that are actually involved in that. 

In other words, you have the comptroller people, you have the 

personnel people, you have the policy people, and you have the acquisition workforce. 

Senator LIEBERMAN. Right. I’m curious as to really what the size 

of the real acquisition workforce is, leaving out the depots and the 

rest. 

Mr. SULLIVAN. I don’t have that answer right now. I know we’re 

doing work on that. 

Senator LIEBERMAN. Good. Get it for the record, please. 

Mr. SULLIVAN. Yes, sir. 

[The information referred to follows:] 

In our recent review of the Department of Defense’s (DOD) acquisition workforce 

management (GAO–09–342), we found that, according to DOD data, the number of 

civilian and military personnel in the Department’s acquisition workforce totaled 

about 126,000 at the end of fiscal year 2007 compared to about 129,000 personnel 

in 2001, a decline of about 2.5 percent. During this same time period, the number 

of contracting actions valued at over $100,000 increased by 62 percent and dollars 

obligated on contracts increased by 116 percent, according to DOD. Moreover, DOD 

84 

has reported that the number of major defense acquisition programs has increased 

from 70 to 95. To augment its declining in-house acquisition workforce, DOD has 

relied more heavily on contractor personnel. However, DOD does not collect or track 

information on contractor personnel, despite their being a key segment of the total 

acquisition workforce. 

Senator LIEBERMAN. There were two other things that there 

seemed to be an interesting consensus on, and they’re quite different, about, you might say, principles for acquisition; a little different than anything we’ve talked about today. They’re not unfamiliar. One was that our original position on acquisition should be 

to favor fixed-price contracts over cost-plus, and to favor competitive bidding, as opposed to negotiated contracts. I want to ask, to 

the extent that we have time—let’s just focus on the fixed-price. 

Our sense, as we discussed at our breakout session with probably 

about 25 people, was, generally speaking, private sector favors competition. So, why are we favoring cost-plus? Does the taxpayer really benefit from that? 

Mr. Adolph, you always get asked last because we’re going left 

to right, so let’s start from the right and ask you about that. 

Mr. ADOLPH. I think the other three panel members have more 

expertise in this issue. But, when you’re in the basic research 

arena, it’s very difficult to do on a fixed-price basis. It should be 

accomplished using cost-plus contracts. 

Once you get beyond research, then the next challenge in the development is system integration. That’s not an insignificant task 

with the very complex systems we’re developing today. But, once 

you get beyond that point then I think you reach a point where you 

can really consider going to fixed-price for downstream procurement. 

Senator LIEBERMAN. Right. That’s interesting. 

Dr. Kaminski, I think, just to clarify—if I could really make it 

too simple—what if we had a law that said, ‘‘Defense contracts 

ought to be done on a fixed-price basis unless the Secretary certifies that there’s a good reason not to’’? 

Dr. KAMINSKI. I think that would end up requiring a lot of certification, Senator Lieberman, for the following reason. 

Senator LIEBERMAN. Right. 

Dr. KAMINSKI. I believe fixed-price contracts are completely appropriate when we know precisely what it is we’re going to buy. If 

there’s uncertainty in what we’re going to buy, and we know we’re 

going to change, and we don’t know yet quite how we’re going to 

change, I think we end up on the wrong end of the bargain negotiating a fixed-price contract and then having to go back and renegotiate that effort for every change that occurs, especially when the 

contractor has already priced in some contingency in the fixed 

price. 

So, there’s a time, for example, in the program, where perhaps 

we are working through this in development, and then we settle in 

on what we want to buy, and we’re ready to enter a well-defined 

production program. That would be a fine time to do a fixed-price 

contract. 

So, I think you have to pass that criteria, knowing what it is you 

want to buy, and have it be predictable. 

85 

Senator LIEBERMAN. Okay. My time is running out, but, Mr. Sullivan, Dr. Gansler, if you’d give me a quick answer to a big question and follow up with writing. 

Mr. SULLIVAN. For major developments, I think it would be very 

difficult to go to a fixed-price contract for that because of the unknowns that are involved. But, I would say that if you work on requirements and try to do some of the things that we’ve been talking about here today, in terms of staying in what is doable, and 

having shorter cycle times to get these things done, you could have 

cost-plus development contracts that don’t get so out of control. It 

really goes back to how much knowledge you have when you set 

out. 

Senator LIEBERMAN. Okay. 

Dr. Gansler? 

Dr. GANSLER. I think that it’s a question of what’s the meaning 

of a ‘‘fixed-price contract’’? As Senator McCain said, you get 75 

changes every week. The contract continues to change hourly, in effect. I think it’s very clear, when you have a stabilized and lowerrisk program, that a fixed-price makes a lot of sense, it does give 

an incentive for the contractors. On the other hand, the cost-plus, 

I would say, we haven’t been using the incentives that are available with the cost-plus-type contracts as well as we should, and I 

think, clearly, for research-and-development-type activities, costplus is an appropriate way to do it, but the ‘‘plus’’ part is an incentive rather than a fixed fee, I think. I would use the incentive 

more. 

Senator LIEBERMAN. Those are very helpful answers. You encourage me to think that we ought to take a look, not at fixed prices 

on across-the-board answer, but to apply it by some selective 

means, and that, in doing so, we might benefit the taxpayers. 

Thanks very much. 

Chairman LEVIN. Thank you, Senator Lieberman. 

Senator Webb. 

Senator WEBB. Thank you, Mr. Chairman. I’d like to add my appreciation for your taking on this issue. As has been so clear today, 

we’re burning up a lot of money, and we’re not getting a lot of product right now, particularly in the shipbuilding programs and aircraft programs. 

Dr. Gansler, when you’re talking about people, which everyone 

seems to agree is the major issue, I was thinking about all the different years and different positions I’ve had on different sides of 

the table, here, working on this issue. It’s so clear that what we 

need is disciplined management, not only on the people side, but 

in the system itself. ‘‘People’’ include people in government, on this 

side of government, it includes people in business. We have challenges because there are not a lot of people in the military who use 

the business concepts, quite frankly, and they’re asked to manage 

these programs. There are not a lot of businesspeople who are used 

to how product comes through a governmental system. 

I believe it was Mr. Sullivan who mentioned the creation of the 

Under Secretary of Defense for Acquisition in 1986. I was actually 

on Caspar Weinberger’s staff when that position was created. We 

had a very talented individual who came into the position. He was 

bewildered with all the different steps that were required to get a 

86 

system through the process. It’s something you just don’t see in the 

normal business world. There are lots of checks and balances. Some 

of them are appropriate, and some of them are less than appropriate. 

Dr. Gansler, I found your analogy with the LCS brought back a 

lot of different memories. They say that a camel is a horse created 

by a committee. I can go all the way back to the M–16, when they 

first developed the M–16, and they developed it directly toward a 

jungle environment, and then the different requirements were put 

on it to be able to be used in the desert and other different places. 

They put a different round in it that carboned up the chamber so 

people were dying in Vietnam because the weapon system requirements had changed as it evolved. 

I was in DOD when we were trying to do the Bradley fighting 

vehicle, and there were different requirements put on it here in 

Congress, so that it was very similar to what you were talking 

about with the LCS. They were saying it should perform different 

functions from the original design, and then there were all these 

press reports about the Bradley fighting vehicle falling over when 

it was going through a water obstacle because it got top heavy. 

Or the FG–7s, the USS Stark-class ships, which were designed 

to build to cost. So, we have fixed cost that we were going to build 

a ship toward, and then you go inside one of these FG–7s, you 

could plink the bulkheads on a FG–7, they were so thin. So when 

an Exocet missile hit the USS Stark in the Persian Gulf when I 

was Secretary of the Navy, it went all the way through the ship 

because they had had to make adjustments based on the cost rather than on some other areas. 

So it’s a very complicated question. I think the key, when I look 

back, is if you can find the right leadership at the top on a program, negotiate and agree on general requirements—there are always going to be fixes—we’ll get a program through. Probably the 

best example of that is when they put Al Gray, who later became 

Commandant of the Marine Corps, and he had the Development 

Center, they put him on the Light Armored Vehicle, and he got 

that program through in about a year. He just pushed it through, 

made all the negotiations, was very firm with people over here in 

Congress, as to what the requirements were for the Marine Corps, 

and it was a very successful program. 

I have one question I would like to put in front of you because 

I’m very concerned about it, and that is the state of all these programs in the United States Navy right now. You talk about the Polaris as having been probably the best analytical prototype of how 

to build a weapon system. One of the things about our submarine 

programs is that we built the frame, and then we added the technology onto the frame, similar to, say, the C–130, rather than continually building a new frame with all the costs and the time that 

goes into that. I’ve just been really struck over the last couple of 

years at how difficult the Navy procurements appear to be, and I’m 

trying to get my arms around why. I would be happy to hear from 

any of you who would like to begin answering that question. 

Dr. KAMINSKI. I’ll make one comment. I think a key thing that 

impacts the programs are stability. So, if you see a program that 

we’ve had in production for some period of time, they have very 

87 

good learning curves on those ships when we have a stable production program. Our issues sometimes are with first-ship cost. But, 

if you look at the cost of subsequent ships, what’s happening there 

is very competitive with commercial-kind of production experience. 

Where we’re producing ships regularly—one of the ship families in 

which we’re doing that is the DDG–51. We have two yards. We 

have some competitive arrangements. Not quite head-to-head competitions, but there are some incentives in those programs. But, it 

is a well-planned, stable program that we’ve been producing. I 

think that approach would benefit us. It’s the areas where we’ve 

had instability where we’ve more problems. 

Dr. GANSLER. Yes, I guess I would approach it by thinking about, 

‘‘What is it the Navy really needs for the 21st century?’’ and what 

types of ships they’re going to want. There is a resistance to 

change. Take the Arsenal ship, for example, which was primarily 

to support the Marines and the Army onshore, it was resisted significantly in terms of it being a low-cost ship for few people. The 

highest costs in the Navy are for people and fuel if you look at the 

life-cycle costs of a ship. So trying to drive down the number of people on the ship and improve the fuel utilization are things we need 

to stress. Those are not the traditional things that are emphasized 

in the Navy construction of ships. I think it’s a different look that 

we need to think about. The LCS has the same thing. Is that something the Navy really wanted or really resisted? So it’s more the 

institutional inertia that has to be changed, I think, in terms of 

what the future Navy for the 21st century is going to need. 

Senator WEBB. Do any of you see this as leadership failures in 

the Navy? [No response.] 

In terms of defining these objectives? 

Dr. GANSLER. There have been some real successes. The F–18E/ 

F on the Navy program was extremely well-managed, but that was 

because they had some really top people doing it, they had a clear 

objective, it was a incremental version of a prior demonstrated program, and it was well done. 

Another big success is the patrol frigate. It kept up competition 

throughout its life, and it had the steepest learning curves of any 

ship in the Navy. So, there are some success stories, but I think 

lessons learned haven’t been widely applied. 

Senator WEBB. Thank you, Mr. Chairman. My time’s up. 

Chairman LEVIN. Thank you, Senator Webb. 

Any other questions of this panel before we excuse them? [No response.] 

Thank you all for your time. Some of you came some distance to 

get here, at some inconvenience. At least one of you had to give up 

a family commitment, and we won’t identify who that was because 

the family is better off not knowing, maybe. [Laughter.] 

But we’re very grateful to all of you for your testimony. It’s very, 

very helpful. 

I will also submit for the record the text of the Weapon Systems 

Acquisition Reform Act of 2009; a statement from Ken Krieg, 

former Under Secretary of Defense for Acquisition, Technology, and 

Logistics; a statement from Moshe Schwartz from the Congressional Research Service; and the DOD Inspector General Acquisition Workforce Count Report. 

88 

[Additional material included for the record.] 


303ful22.eps
89 


303ful23.eps
90 


303ful24.eps
91 


303ful25.eps
92 


303ful26.eps
93 


303ful27.eps
94 


303ful28.eps
95 


303ful29.eps
96 


303ful30.eps
97 


303ful31.eps
98 


303ful32.eps
99 


303ful33.eps
100 


303ful34.eps
101 


303ful35.eps
102 


303ful36.eps
103 


303ful37.eps
104 


303ful38.eps
105 


303ful39.eps
106 


303ful40.eps
107 


303ful41.eps
108 


303ful42.eps
109 


303ful43.eps
110 


303ful44.eps
111 


303ful45.eps
112 


303ful46.eps
113 


303ful47.eps
114 


303ful48.eps
115 


303ful49.eps
116 


303ful50.eps
117 


303ful51.eps
118 


303ful52.eps
119 


303ful53.eps
120 


303ful54.eps
121 


303ful55.eps
122 


303ful56.eps
123 


303ful57.eps
124 


303ful58.eps
125 


303ful59.eps
126 


303ful60.eps
127 


303ful61.eps
128 


303ful62.eps
129 


303ful63.eps
130 


303ful64.eps
131 


303ful65.eps
132 


303ful66.eps
133 


303ful67.eps
134 


303ful68.eps
135 


303ful69.eps
136 


303ful70.eps
137 


303ful71.eps
138 


303ful72.eps
139 


303ful73.eps
140 


303ful74.eps
141 


303ful75.eps
142 


303ful76.eps
143 


303ful77.eps
144 


303ful78.eps
145 


303ful79.eps
146 


303ful80.eps
147 


303ful81.eps
148 


303ful82.eps
149 


303ful83.eps
150 


303ful84.eps
151 


303ful85.eps
152 


303ful86.eps
153 


303ful87.eps
154 


303ful88.eps
155 


303ful89.eps
156 


303ful90.eps
157 


303ful91.eps
158 


303ful92.eps
159 


303ful93.eps
160 


303ful94.eps
161 


303ful95.eps
162 


303ful96.eps
163 


303ful97.eps
164 


303ful98.eps
165 


303ful99.eps
166 


303fu100.eps
167 


303fu101.eps
168 


303fu102.eps
169 


303fu103.eps
170 


303fu104.eps
171 


303fu105.eps
172 


303fu106.eps
173 


303fu107.eps
174 


303fu108.eps
175 

Chairman LEVIN. We’ll stand adjourned. 

[Questions for the record with answers supplied follow:] 

QUESTIONS SUBMITTED BY SENATOR JOSEPH I. LIEBERMAN

STATUS OF CURRENT ACQUISITION SYSTEM

1. Senator LIEBERMAN. Mr. Sullivan and Dr. Gansler, I have several concerns 

with the acquisition system as it currently operates. Most importantly, it seems that 

the requirements validation process under the Joint Capabilities Integration and 

Development System (JCIDS) is not synchronized with the Planning, Programming, 

Budgeting, and Execution (PPBE) funding process. Can any effort at reform succeed 

without addressing this fundamental flaw in acquisition management? 


303fu109.eps
176 

Mr. SULLIVAN. We do not believe that there can be success until there is better 

synchronization between the Department of Defense’s (DOD) requirements, funding, 

and acquisition processes. The Government Accountability Office (GAO) has previously reported that the lack of integration between the JCIDS and PPBE processes is a key factor that contributes to the Department’s inability to achieve a balanced portfolio of weapon system programs that meets the needs of the joint 

warfighter and matches needs with available resources. The JCIDS process has not 

been effective in prioritizing needs from a joint, Department-wide perspective, and 

it largely approves capability needs without accounting for the resources or technologies that will be needed to acquire the desired capabilities. Resource allocation 

decisions in DOD take place through the PPBE process, which is separate from 

JCIDS. PPBE largely allocates resources on a Service-by-Service basis and does not 

effectively link resources to capabilities. In addition, the process allows too many 

programs to start development with unreliable cost estimates and without a commitment to fully fund them. Until DOD establishes a more integrated approach to 

weapon system acquisition, it will continue to struggle to effectively prioritize 

warfighting needs, make informed trade-offs, and achieve a balanced mix of weapon 

systems that are affordable, feasible, and provide the best value to the warfighter. 

Dr. GANSLER. I believe ‘‘cost’’ should be an essential part of the ‘‘requirements 

process’’—since, in a resource-constrained environment, it directly determines ‘‘numbers’’ (of systems procured) and that determines force effectiveness (even more than 

individual weapon’s effectiveness). 

2. Senator LIEBERMAN. Mr. Sullivan and Dr. Gansler, what are your recommendations for effectively balancing the requirement validation process with the calendardriven funding process? 

Mr. SULLIVAN. To improve DOD’s ability to deliver weapon systems at the right 

time and right cost, GAO has recommended that the Department implement an enterprise-wide portfolio management approach to making weapon system investments 

that integrates the determination of warfighting needs with available resources and 

cuts across the Services by functional or capability areas. Such an approach would 

focus on assessing weapon system investments collectively from an enterprise level, 

rather than as independent and unrelated initiatives. By following a disciplined, integrated process—during which the relative pros and cons of competing weapon system proposals are assessed based on strategic objectives, warfighter needs, and 

available resources, and where tough decisions about which investments to pursue 

and not to pursue are made—DOD could minimize duplication between Service components and more effectively support each new development program it commits to. 

Furthermore, to ensure effective weapon investment decisions are made, GAO has 

recommended that a single point of accountability be established at the Department 

level with the authority, responsibility, and tools to implement portfolio management. 

Dr. GANSLER. My answer to question #1 (above) will greatly aid in this; but, in 

general, the ‘‘requirements process’’ must look well beyond the near-term budget 

(funding) process. Nonetheless, the frequent budget changes (particularly, by Congress) introduce great instability and inefficiency into the implementation of the 

weapons acquisition process. 

JOINT REQUIREMENTS

3. Senator LIEBERMAN. Mr. Sullivan and Dr. Kaminski, I am concerned that DOD 

has historically done a poor job of procuring capabilities that address joint requirements, and more specifically, that the Services have little incentive to spend what 

they see as their limited budgets on capabilities that will principally benefit sister 

Services. One particular instance of this is the Multi-Platform Radar Technology Insertion Program, and the Air Force’s apparent reluctance to procure a system that 

will principally serve the Army’s battlefield management requirements. How can the 

acquisition process better prioritize and procure such crucial joint capabilities? 

Mr. SULLIVAN. GAO has recommended that DOD should determine and allocate 

appropriate resources for more effective joint capabilities development planning. The 

functional capabilities boards, which were established to manage the JCIDS process 

and facilitate the prioritization of needs, have not been staffed or resourced to effectively carry out these duties. Similarly, the combatant commands (COCOMs) also 

lack the analytic capacity and resources to become more fully engaged in JCIDS. 

GAO recently reviewed JCIDS documentation related to new warfighting capabilities and found that most—almost 70 percent—were sponsored by the military Services with little involvement from the joint community, including the COCOMs, 

177 

which are responsible for carrying out military missions. The Services drive the determination of capability needs, in part because they retain most of DOD’s analytical capacity and resources for requirements development. 

Within the Department’s PPBE process, the individual military Services are responsible for budgeting and allocating resources under authority that is commonly 

understood to be based on title 10 of the United States Code—to organize, train, 

and equip military forces. In this structure, the budget is based more on individual, 

service-focused needs than on joint warfighting needs. In the past, GAO has reported that this situation has contributed to interoperability problems among weapon systems and unnecessary duplication of capabilities in some areas. The Office of 

the Secretary of Defense (OSD) reviews and makes adjustments to the Services’ 

budgets, but this takes place toward the end of the PPBE cycle when it can be difficult and disruptive to make changes, such as moving funds to higher-priority programs or to support joint needs. DOD has recently taken steps to establish capability portfolio management in selected areas in an attempt to overcome the servicecentric nature of the resource allocation process. However, because the portfolios are 

largely advisory and lack authority and control over the allocation of resources, 

there may be limits to achieving joint capabilities. 

Dr. KAMINSKI. The priority must be set in the budget process, with key oversight 

from OSD and the Joint Chief of Staff. The procurement can be improved by first 

developing (a several year process) then assigning key program management personnel with the requisite training and domain experience. Alignment of the responsibility, authority, and accountability of the program manager will require that a 

degree of trust be established between the program manager and those responsible 

for our oversight mechanisms. We must be prepared to delegate authority to the 

program manager, and provide him or her with some flexibility to manage—to adjust levels and allocation of funding, to adjust the allocation of performance parameters, to adjust schedule, and to tailor the acquisition approach to be responsive to 

the need. Clearly, there must be bounds established beyond which the program 

manager must seek approval from oversight authorities. But I believe these bounds 

are too narrow and inflexible today. We also need to provide sufficient upfront funding, and maintain funding stability throughout program execution. 

4. Senator LIEBERMAN. Mr. Sullivan and Dr. Kaminski, are either the JCIDS or 

PPBE as currently constructed able to overcome parochial service interests in this 

regard? 

Mr. SULLIVAN. No, I do not believe that the JCIDS and PPBE processes are constructed effectively to address the service-centric manner in which weapon systems 

are acquired in DOD. Past studies chartered by DOD and other organizations have 

reported similar conclusions and made a number of recommendations to improve the 

acquisition of joint warfighting capabilities. 

Dr. KAMINSKI. Both help, but I don’t believe they are sufficient. 

DEPARTMENT OF DEFENSE SCIENCE AND TECHNOLOGY PROGRAM

5. Senator LIEBERMAN. Dr. Gansler and Dr. Kaminski, as we try to reduce risk 

in acquisition programs, what role does (or should) DOD science and technology 

(S&T) program play? 

Dr. GANSLER. S&T are critical to both ‘‘staying ahead’’ (which is the basis of our 

national security strategy of ‘‘technological superiority’’) as well as to risk reduction—since we need to prove out the S&T first, before introducing it into weapons 

acquisition. 

Dr. KAMINSKI. With good S&T programs supporting development planning, we can 

assign managers to develop prototypes, critical systems or components needed to 

better understand cost/performance trades and reduce risk. With the appropriate 

technology base in place, it is reasonable to expect that many of these developments 

can be completed in 2–4 years, so one manager will be in place from start to delivery of those critical subsystems during the development planning phase. This should 

allow the full system development to proceed on a shorter schedule as a result of 

the risk reduction. 

6. Senator LIEBERMAN. Dr. Gansler and Dr. Kaminski, do you think that the Department’s S&T programs, and in particular the DOD laboratories, are sufficiently 

resourced and staffed to provide support to the acquisition programs and address 

the problems you are discussing today? 

Dr. GANSLER. The DOD’s S&T program must be adequately funded to ‘‘stay 

ahead’’; and, traditionally, whenever there is a shortage of DOD money it is one of 

178 

the areas the Services tend to cut (since it is long-term). Thus, in the likelihood of 

a coming budget crunch, it must be protected—not just in the DOD labs, but particularly in the universities and in industry. 

Dr. KAMINSKI. No. We are short in both quantity and quality of staffing, and are 

not using the latest tools to support early modeling, simulation, and testing. 

7. Senator LIEBERMAN. Dr. Gansler and Dr. Kaminski, what recommendations do 

you have for this committee to improve the quality of the DOD S&T program and 

the quality of the DOD laboratories? 

Dr. GANSLER. Adequate funding, and allowance for foreign graduate students, 

scholars, and researchers (per Presidential Decision Directive-189) to work on DOD 

S&T, along with ‘‘peer review’’ of all proposals, are critical to the United States staying ahead. The SBIR and STTR programs are also a great benefit, and must be 

maintained. 

Dr. KAMINSKI. The first step is to ensure that we not only restore, but also enhance early and continuing systems engineering coupled with the effective development planning needed to drive our S&T program. This will require commitment of 

more significant investment dollars earlier in our acquisition programs, and a commitment to build a cadre of systems engineers and development planners with the 

education, training, and domain experience needed to be effective. Attracting ‘‘best 

and brightest’’ to this work—and keeping them—will require a personnel system 

that will identify and track these important human resources and establish a career 

path to allow those who are successful to advance to senior program management 

and leadership positions. Their domain experience will be enhanced by managing 

the building of critical technology demonstrators and subsystems during the development planning program, reducing risk, and building skills and experience at the 

same time. 

QUESTION SUBMITTED BY SENATOR JACK REED

S. 454 LEGISLATION

8. Senator REED. Dr. Gansler, in the proposed S. 454 legislation, section 203 focuses on maximizing competition throughout the life cycle of major acquisition programs. One proposed provision is that modular, open architectures are used to enable competition for upgrades. Please comment on how the implementation of this 

recommendation would positively impact major weapons programs that have been 

sole sourced for decades, and how the Services can rapidly implement these changes 

to achieve maximum benefits. In addition, please comment on the feasibility of applying this provision to current programs, not just new programs. 

Dr. GANSLER. Let me begin by noting that the objective should not be ‘‘maximizing 

competition’’; rather, it should be ‘‘maximizing the effectiveness of competition.’’ 

Competition for its own sake, can be detrimental. For example, there must always 

be the ‘‘option’’ of competition, but if a firm is constantly improving performance and 

lowering costs, they should be rewarded by not recompeting the program; but there 

must always be the option of introducing competition if costs are rising (so the 

‘‘threat’’ of competition is what creates the incentive for lowering costs; and the follow-on is the reward). 

Open architectures are clearly one way to allow competition to be introduced at 

the subsystem level (which often is 70–80 percent of the total weapons cost); so it 

should be greatly encouraged—both on upgrades of current systems and on new 

ones. 

QUESTIONS SUBMITTED BY SENATOR DANIEL K. AKAKA

WEAPON SYSTEMS ACQUISITION REFORM ACT

9. Senator AKAKA. Mr. Sullivan, section 105 of the Weapon Systems Acquisition 

Reform Act recommends the Joint Requirement Oversight Council (JROC) seeks 

input from the combatant commanders in identifying joint military requirements. 

Currently, the combatant commanders produce an annual Integrated Priority List 

(IPL) that attempts to satisfy this same goal. In your prepared testimony, you stated some of the combatant commanders do not feel their needs are sufficiently addressed. What must be done within DOD to fully implement the intent of this legislation to ensure combatant commander’s influence throughout the acquisition system to decrease the Service-centric environment we’ve seen for years? 

179 

Mr. SULLIVAN. GAO and several other studies have recommended that DOD increase joint analytic resources for a less stovepiped understanding of warfighting 

needs. While each COCOM submits an IPL to the Chairman of the Joint Chiefs of 

Staff each year, which defines the COCOM’s highest-priority needs, it does not appear that the COCOM needs are well integrated with JCIDS. According to officials 

from several COCOMs that we have met with, needs identified through IPLs are 

not typically developed into JCIDS capability proposals. Some COCOM officials 

pointed out to us that because of their limited resources, they must pick and choose 

capability needs and persuade one or more of the military Services to sponsor a proposal that addresses the COCOM’s need. The Department has been working to give 

more time and priority to the commanders’ IPLs, but more attention is needed. 

10. Senator AKAKA. Mr. Sullivan, in your prepared testimony, you mentioned the 

establishment of the DOD capability portfolio management process. DOD identified 

nine joint capability area portfolios to mitigate the tendency to develop Service-centric, stovepiped solutions. However, the capability portfolio managers lack the necessary authority and control required to be successful. How can DOD reform their 

capability portfolio management process to mitigate the tendency to develop Servicecentric, stovepiped solutions? 

Mr. SULLIVAN. DOD’s capability portfolios are intended to advise the Deputy Secretary of Defense on how to optimize investments within their respective capability 

areas. They have no independent decisionmaking authority and must operate within 

the Department’s existing framework for acquiring weapon systems. That is, capability needs are still determined through JCIDS and resources are allocated through 

PPBE. Although it is too soon to assess the impact of capability portfolio management, according to some DOD officials, the portfolios provided key input and recommendations in the 2010 budget process. However, without portfolios in which 

managers have authority and control over resources, DOD is at risk of continuing 

to acquire weapon systems in a stovepiped manner. Ultimately, DOD needs to develop an integrated portfolio approach that brings the determination of requirements together with the allocation of resources. To be effective, such an approach 

must have committed leadership, empowered portfolio managers, and accountability 

at all levels of the Department. 

11. Senator AKAKA. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, changes made 

to a system after acceptance of review cost many times more than the same changes 

made during the requirements gathering phase. How can we improve the requirements gathering phases of the acquisition of major weapons systems to front-load 

these changes to achieve significant cost savings? 

Dr. GANSLER. Please refer to my answer to question #1. If cost is a requirement, 

then changes that come in must not only have showed how they will improve performance but how they will live within the cost requirement. 

Clearly, the earlier this is done, the better; but technology changes, threat 

changes, et cetera, do come along, and as long as they are explicitly within the overall cost requirement they can be allowed in. 

Dr. KAMINSKI. Requirements adjustments during execution of acquisition programs are not necessarily bad if they are conducted within the framework of value 

propositions. Good program management requires continuing adjustment of the requirements flow-down and allocation to major subsystems based upon expected performance, schedule, and cost. Even top level requirements require continuing assessment and adjustment based upon similar value propositions. As we discover more 

about costs and performance during execution, good program managers should be 

interacting with requirements developers to adjust as needed to provide the best 

value for the user. This requires some flexibility in cost allocation, and it also places 

demands upon requirements developers to become involved in tradeoff decisions. To 

be effective, requirements developers as well as acquisition managers must have experience with systems engineering tools and techniques, and both must have sufficient domain experience to engage productively in cost-performance tradeoffs. 

Mr. ADOLPH. A disciplined analysis of alternatives process should be employed to 

support the JCIDS and acquisition processes, from capability needs identification to 

include system design and development as well as life-cycle improvement. Emphasis 

must be placed on improving the processes for relating mission effectiveness and 

cost to system design, system performance, and suitability. Effective feedback processes are of special importance for spiral developments to identify enhancements 

which will improve performance and suitability. Improving the quality and speed of 

this feedback is increasingly important in responding to rapid changes in threat environments. 

180 

12. Senator AKAKA. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, significant cost 

overruns are incurred from the instability of mission and design requirements during testing and even, in some cases, production. To what extent do you think it 

would be advantageous to more rigorously enforce a freeze on fundamental requirements at the system requirements review gate? 

Dr. GANSLER. The greatest causes of cost growth, after program initiation, are 

budget changes (often caused by Congress) and ‘‘requirements’’ changes—both of 

which can be controlled by discipline. (See question #11.) 

Dr. KAMINSKI. It is always beneficial to have a firm understanding of very top 

level requirements at the start of a program. But adjustments of subordinate requirements during execution of acquisition programs are not necessarily bad if they 

are conducted within the framework of value propositions. Even top level requirements need continuing assessment based upon evolving value propositions. As we 

discover more about costs and performance during execution, good program managers should be interacting with requirements developers to adjust as needed to provide best value for the user. This requires some flexibility in cost allocation, and 

it also places demands upon requirements developers to become involved in tradeoff 

decisions. To be effective, requirements developers as well as acquisition managers 

must have experience with systems engineering tools and techniques, and both must 

have sufficient domain experience to engage productively in cost-performance tradeoffs. 

Mr. ADOLPH. Rigorous enforcement of key requirements thresholds should be the 

norm when entering system development and demonstration. There needs to be 

more rigor and discipline in the entire requirements definition and acquisition process. Issues that need to be addressed in relation to requirements setting include 

technology readiness (see response to question 13 below), the translation of requirements into design criteria, with attention to testability at the subsystem and system 

levels, as well as defining thresholds for key performance parameters. 

13. Senator AKAKA. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, cost overruns are 

also caused by the inherent unpredictability of the development of new technologies 

in the course of fulfilling system contracts. Would you recommend restricting the 

proposed designs for weapons systems to proven technologies while adequately funding contractual vehicles for pure research? 

Dr. GANSLER. I would restrict the first ‘‘block’’ of the system to proven technology; 

then—using ‘‘spiral development’’—evolve the system’s future blocks as future technology is proven out; and as long as the future technology either improves performance (at the same cost) or lowers the cost. This future technology should be funded 

in parallel with the system, until it is proven. 

Dr. KAMINSKI. This will be helped by better development planning and alignment 

of incentives. With good development planning, we can assign managers to develop 

prototypes, critical systems or components needed to better understand cost/performance trades and reduce risk. It is reasonable to expect that many of these developments can be completed in 2–4 years, so we do not need to restrict ourselves to 

proven technology at the beginning of development planning. As these critical subsystems are delivered and tested, the risk reduction and domain experience gained 

in both government and industry will allow us to prove the critical technologies involved and reduce the time required to develop, integrate, and test the full system. 

We can rely on the experience gained during development planning to apply informed judgment to adjust requirements to improve value, reduce time, and better 

estimate and manage costs. 

Mr. ADOLPH. Weapon system design should be based on proven technologies. History has shown repeatedly that doing otherwise introduces unacceptable risk in to 

weapons system development which necessarily translates into cost growth and 

schedule delays. Assessing whether a technology is sufficiently mature is currently 

accomplished through the Technology Readiness Levels (TRL) construct which 

should employ government test and evaluation (T&E) as the key component in moving a technology from TRL 4 to TRL 6. Recent consideration has been given to 

strengthening early involvement of the test community by reinvigorating the government developmental test process and by involvement of OT&E in readiness assessments. These are sound initiatives and should be focused on providing rigor to 

the TRL assessments in terms of the test methodology as well as the test environment. Implementation of this more disciplined approach will require a reversal to 

the trend of cutting back on government test personnel as well as strengthening 

their role in supporting the acquisition process. In that regard, the Director of Developmental Test and Evaluation should make an assessment of technological maturity and integration risk, based on test results and report the findings to the Undersecretary of Defense for Acquisition, Technology and Logistics. 

181 

There may be rare exceptions when a requirement is so pressing that risks must 

be taken to design a system which incorporates an unproven technology. In those 

instances, a high-priority, adequately funded, and closely monitored effort must be 

undertaken to mature the technology. This must include an early, rigorous assessment of the difficulties associated with the development effort as well as testing to 

fully assess the requisite maturity. 

Additionally, the TRL process has been focused on the technology of system components. Problems with manufacturing these components in production quantities 

have also led to significant cost growth for weapons systems. Previous legislation 

established the Manufacturing Technology Program to identify and develop initiatives to improve manufacturing quality, productivity and technology. Consideration 

should be given to expanding the TRL process to evaluate the maturity of the production methods by which parts are manufactured. 

QUESTIONS SUBMITTED BY SENATOR BILL NELSON

MODELING AND SIMULATION

14. Senator BILL NELSON. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, computer 

modeling and simulation (M&S) tools can be very useful in discovering problems in 

major systems before we spend a lot of money or time on hardware development. 

DOD has some programs in M&S—which in my opinion are underfunded and disjointed. What is your assessment of M&S activities within DOD and their ability 

to address some of the acquisition problems we are discussing here today? 

Dr. GANSLER. M&S can be a great aid in system design and performance evaluation, and should be an important element in the development of all systems. 

Dr. KAMINSKI. M&S can have a very significant positive impact on acquisition performance if there is sufficient emphasis on continuing validation of M&S tools and 

integration with T&E. Giving early and serious attention T&E will require strengthening our T&E organizations and personnel. T&E is often an afterthought, and contracts are often written without any mention of how we will test the product. We 

spend large amounts of money when a large development team waits for test results. The alternative is to spend less money and time by early development of M&S 

tools, continuously validating the models with testing at critical points, and considering the investment in M&S and test resources as a critical part of our systems 

engineering and development planning efforts. 

Mr. ADOLPH. With some exceptions, the capabilities and limitations of simulations, simulators and physics-based models of individual subsystems are well understood. These tools are generally used where appropriate. However the use of modeling to support requirements definition, analysis of alternatives, and test design 

needs to be pursued more vigorously. The expectations regarding the use of M&S 

to support elements of the acquisition process, including system integration, T&E, 

and manufacturing have increased along with advances in computer technology. 

Distributed T&E (and training) using multiple ranges, laboratories, and facilities 

have benefited from the use of live, virtual, and constructive simulations. However, 

the availability of adequate models and modeling tools involving human dynamics 

for use in T&E, mission planning, systems engineering, and risk assessment has not 

kept pace. 

M&S to support acquisition/test programs has been the norm for decades, and received additional impetus in the 1990s with advances in information processing 

technology. In many cases, the incremental improvements as related to acquisition 

were oversold in the late 1980s and 1990s. Additionally, the requirements for 

verification, validation, and accreditation have proven to be more costly and 

timeconsuming than anticipated. Many of the M&S activities initiated in the last 

15 years were disjointed, with few successful attempts to coordinate requirements, 

build and share common models, correlate performance, and ensure that the models 

were adequate for the intended applications. Numerous recent M&S developments 

have been terminated after spending millions because of the failure to produce a 

useful product. 

Building models to meet every need is not feasible. An overall M&S vision is required to identify where efforts should be focused, and to ensure that new efforts 

are within that vision. If high priority M&S efforts within that vision could be identified, it would be possible to establish requirements, understand what is currently 

available, and then determine shortfalls in M&S capability. One approach is exemplified by the Testing in a Joint Environment Roadmap, which was recently developed by DOT&E and approved by the Deputy Secretary of Defense. While focused 

on T&E usage, this capability would meet a significant percentage of contractor deVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00185 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
182 

velopment and testing M&S requirements. The T&E roadmap identifies a requirement for a distributed live, virtual, constructive T&E capability; largely supported 

by existing models of friendly and threat systems. Much of this capability could 

come from existing models of acquisition systems developed by program contractors, 

in conjunction with threat models available from the intelligence agencies. However, 

there has been virtually no effort to identify requirements for high priority joint 

missions, to determine what friendly and threat force representations are required, 

assess the availability of existing models that meet those requirements, and then 

use that information to define M&S shortfalls. Additionally, there are insufficient 

mechanisms and processes to feed back data from operational tests and field exercises to further validate and refine models. Archiving mechanisms that can store 

and locate data for future applications are needed. 

There are also supporting M&S requirements (engineering and physics-based 

models) which are required in the validation of higher-level engagement models. 

Higher level models, such as envisioned in the T&E roadmap, could be used to identify areas with the greatest risk and uncertainty to better define areas where engineering and physics-based models are required. The initial effort should be to review 

the past studies on M&S in the DOD and determine those requirements that could 

contribute the most to improving M&S capability. While most studies have identical 

or similar recommendations, the majority of those recommendations have not been 

implemented. 

15. Senator AKAKA. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, how would you 

recommend addressing shortfalls in DOD activities in M&S? 

Dr. GANSLER. To address shortfalls, I would ask an independent group (such as 

the Defense Science Board) for an assessment and a set of recommendations (including appropriate funding). 

Dr. KAMINSKI. We need upfront investment in M&S tools to understand the value 

propositions and the tradeoffs among cost, schedule, and performance of the mission. We spend large amounts of money when a large development team waits for 

test results. The alternative is to spend less money and time by early development 

of M&S tools, continuously validating the models with testing at critical points, and 

considering the investment in M&S and test resources as a critical part of our systems engineering and development planning efforts. 

Mr. ADOLPH. Focused M&S business plans must be developed. Separate plans are 

needed for requirements definition, acquisition/test, training and force structure 

evaluations. As discussed above, the latter plans should be used to identify critical 

joint missions and provide a basis for prioritizing M&S requirements; to understand 

current capabilities and identify major shortfalls. 

An issue that needs to be addressed is the availability of existing models to meet 

current requirements. Many models were developed by system contractors to meet 

specific needs. In some cases they are proprietary and there are rarely provisions 

for models to be maintained current so they represent a system in the field with 

normal improvements and modifications, or changes to the treats. Often contractors 

are not funded to make their models generally available. The government must address issues relating to long-term configuration control, upgrades, ownership, and 

funding throughout the system lifecycle, to include training. In addition, addressing 

the requirements for a single joint mission doesn’t capture requirements across several potential joint missions and scenarios. 

However, it would provide a starting point for future requirements and identify 

the most sensitive issues in implementation of the overall concept. Addressing one 

major joint mission scenario would identify a large percentage of requirements for 

other joint missions. 

Finally, an important consideration is the physical location of major simulation 

laboratories and related facilities. There are often valid reasons to locate a simulation capability at a contractor development facility to support initial development; 

however it places limitations on the future utility and accessibility of the simulation, 

as well as the need to replicate parts of the simulation capability at a government 

test facility or logistics center. Prior to making a decision regarding the location of 

simulation facilities for each major program, an assessment should be made as to 

the most cost-effective locations(s), taking into account follow-on and future programs. This decisionmaking process should include the appropriate Service MRTFB 

representatives, as well as the TRMC. (See also my response to question #17.) 

183 

TEST AND EVALUATION ENTERPRISE

16. Senator AKAKA. Mr. Adolph, a major step in developing weapons systems is 

ensuring that they are appropriately tested during their development and before 

their operational use. I believe that DOD is letting its testing enterprise decline precipitously in terms of testing equipment, infrastructure, and workforce. What role 

do you feel that the DOD testing enterprise plays in ensuring acquisition programs 

deliver their systems on schedule and on budget? 

Mr. ADOLPH. The role of the DOD testing enterprise is addressed in detail in the 

recent Defense Science Board Task Force Report on Developmental T&E. My written testimony covers the major findings and recommendations from that study, and 

while it emphasizes the impacts of reliability, availability, and maintainability on 

life-cycle costs, the implementation of some acquisition reform initiatives diminished 

the role of government T&E in the acquisition process. 

A brief summary response follows. The traditional role of the government during 

the test planning phase included the identification of test resource requirements and 

test facilities, the development of a test strategy and detailed T&E plans. When programs moved from the planning phase to the execution phase, the government traditionally performed operational assessments, participated in test conduct and analysis, and performed an evaluation of the test results for the government Program 

Office. With a few exceptions, this is no longer the case. In many instances the government testing enterprise has gone from vigorous test participation to oversight to 

out-of-sight. 

As a minimum, government test organizations should reconstitute and retain a 

cadre of experienced T&E personnel to perform the following functions: 

• Participate in the translation of operational requirements into contract 

specifications and key performance parameters 

• Participate in RFP preparation and source selection, including assurance 

that contractor developed models and simulations are available to the government 

• Participate in technology maturity determination and prototype evaluations 

• Perform operational assessments in conjunction with the systems engineering process 

• Participate in developmental T&E planning and technical reviews 

• Participate in test conduct, analysis, reporting capabilities, and limitations, with emphasis on government analysis and reporting 

17. Senator AKAKA. Mr. Adolph, what is your assessment of the state of the DOD’s 

testing facilities and their ability to perform their designated missions? 

Mr. ADOLPH. The focus of DOD’s testing facilities is the Major Range and Test 

Facility Base (MRTFB) which is comprised of those ranges and facilities designated 

most critical to supporting the T&E needs of DOD development and acquisition programs. Since the 1970s, DOD’s intent has been to ensure the capabilities and capacities of these MRTFB facilities/ranges meet the T&E requirements of emerging technologies and weapon system programs, taking into consideration the long-lead times 

necessary to acquire these capabilities, which often dictates that new test technologies be developed and matured. Likewise, a skilled T&E workforce must be in 

place to address the challenges of testing and evaluating new weapon systems with 

increasing complexities of technologies, integration, and missions. Given that this is 

the intention of the MRTFB, where do we stand today? 

As I noted in my written statement, several changes resulted from the implementation of acquisition reform initiatives in the late 1990s. Among them was a de-emphasis on the use of government test facilities, testers, and evaluators. When programs need to use government test facilities, the first inclination has sometimes 

been to use non-MRTFB capabilities to avoid reimbursement costs required by the 

MRTFB charge policies, often using training ranges having little or no T&E infrastructure. If the MRTFB facilities/ranges are used, programs typically prefer to request only the raw data to perform their own evaluations. Consequently, the Services have been reducing their budgets and T&E workforce for sustaining the 

MRTFB. Not only has the MRTFB experienced loss of a large number of the experienced subject matter engineering experts, scientific, and mathematical personnel, 

several MRTFB test facilities are being mothballed or closed. The government skills 

to restore or replace them will be lost over time. 

Investments in new test capabilities in the MRTFB for anticipated weapon system 

technologies and test programs have been increasingly awaiting the first program 

to need them for funding rather than using T&E institutional funds. This practice 

is counter to the intent of having an MRTFB; i.e., anticipating and having T&E caVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00187 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
184 

pabilities in place when new development or acquisition programs require them. 

Some new T&E capabilities require long-lead times to plan, budget, and to develop 

to be ready when needed by a program, thus can cause the test program schedules 

to slip, inadequate capabilities to be accepted as work-arounds, or not to be tested 

at all. Furthermore, these facilities are normally specialized for the programs paying 

for them, and may have little or no residual use for other programs, particularly 

when they are co-located with the prime contractor’s manufacturing facility. While 

the Test Resources Management Center (TRMC) ‘‘Strategic Plan for DOD Test and 

Evaluation Resources’’ serves as a good guide to the actions required to sustain a 

capable MRTFB, the TRMC has virtually no power to require the military departments to sustain adequate operating or investment funding beyond the funds managed through the Central Test and Evaluation Investment Program and T&E/S&T 

Program. As a result, my assessment is that in some instances, the MRTFB no 

longer meet its intended purpose of having adequate capabilities and capacity available for future acquisition programs. 

QUESTIONS SUBMITTED BY SENATOR E. BENJAMIN NELSON

FISCAL RESPONSIBILITY

18. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, cost overruns and fiscal irresponsibility are great concerns for me. During his 

hearing in January, the Secretary of Defense stated that many programs that cost 

more than anticipated are built on an inadequate initial foundation, which in my 

opinion begins with defining requirements. It has been reported that many of the 

COCOMs do not believe their requirements are sufficiently represented by the current process, which seems to be centered on each Service, not the Armed Forces as 

a whole. This results in duplication of capabilities, non-alignment of requirements 

to capability, and interoperability problems, all of which lead to expensive requirements creep. Should COCOMs play a lead role in defining needed capabilities? 

Mr. SULLIVAN. Yes, we believe that the COCOMs should play a stronger role in 

defining capability needs. Requirements determination in DOD, particularly for 

major weapon system programs, continues to be driven largely by the military Services. Our work has revealed that although the COCOMs—which are responsible for 

planning and executing military missions—are the principal joint warfighting customer in DOD, they have played a limited role in determining requirements. In analyzing requirements documentation related to new capability proposals, we recently 

reported that most—almost 70 percent—were sponsored by the military Services 

with little involvement from the COCOMs. Other studies have also raised concerns 

that the Services and the COCOMS do not routinely collaborate to identify possible 

joint capability solutions. 

• The Defense Acquisition Performance Assessment Panel concluded that 

JCIDS resulted in capabilities that did not meet warfighter needs in a timely manner and recommended that JCIDS be replaced with a COCOM-led 

requirements process in which the Services and defense agencies compete 

to provide solutions. 

• The Defense Science Board similarly reported that JCIDS has not provided for increased warfighter influence, but instead actually suppresses 

joint needs in favor of military Service interests, and recommended an increase in the formal participation role of the COCOMs in the JCIDS process. 

• The Center for Strategic and International Studies has also recommended 

that the Joint Forces Command take the lead in conducting capabilities development planning for the COCOMs and become a formal member of the 

JROC. 

By continuing to rely on capability needs defined primarily by the Services, DOD 

may be losing opportunities for improving joint warfighting capabilities and reducing the duplication of capabilities. 

Dr. GANSLER. The Packard Commission and (subsequently) Goldwater-Nichols 

stated that all requirements were ‘‘joint’’ (since that is the way we fight). The JROC 

was set up to represent the COCOMs in the ‘‘requirements process.’’ Interoperability 

should always be a critical design and test requirement. The COCOMs should be 

the source of requirements for both ‘‘urgent needs’’ and for JCTDs. 

Dr. KAMINSKI. I believe that the COCOMs should play a strong role. But to do 

so, they need to develop the expertise needed to engage in cost, schedule, and performance tradeoffs. Even top level requirements need continuing assessment based 

upon evolving value propositions. As we discover more about costs and performance 

185 

during execution, good program managers should be interacting with requirements 

developers to adjust as need to provide best value for the user. This requires some 

flexibility in cost allocation, and it also places demands upon requirements developers to become involved in tradeoff decisions. To be effective, requirements developers as well as acquisition managers must have experience with systems engineering tools and techniques, and both must have sufficient domain experience to engage productively in cost-performance tradeoffs. 

Mr. ADOLPH. I don’t have the background or experience to respond to this question. 

19. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, how successful are DOD’s processes for matching warfighter needs with resources? 

Mr. SULLIVAN. GAO has previously reported that DOD’s inability to achieve a balanced portfolio of weapon system programs that matches joint warfighting needs 

with available resources is due in part to the fragmented structure of the Department’s requirements, funding, and acquisition processes. The JCIDS process has not 

been effective in prioritizing needs from a joint, Department-wide perspective, and 

it largely approves capability needs without accounting for the resources or technologies that will be needed to acquire the desired capabilities. Resource allocation 

decisions in DOD take place through the PPBE process, which is separate from 

JCIDS. PPBE largely allocates resources on a Service-by-Service basis and does not 

effectively link resources to capabilities. In addition, the process allows too many 

programs to start development with unreliable cost estimates and without a commitment to fully fund them. Until DOD establishes a more integrated approach to 

weapon system acquisition, it will continue to struggle to effectively prioritize 

warfighting needs, make informed trade-offs, and achieve a balanced mix of weapon 

systems that are affordable, feasible, and provide the best value to the warfighter. 

While DOD recently reported that the JROC is doing more to seek out and consider input from the COCOMs through regular trips and meetings to discuss capability needs and resourcing issues, we found that many of the COCOMs still do not 

believe their needs—reflected through the IPL process—are sufficiently addressed 

through DOD’s requirements process. In order to grant the COCOMs more influence 

in identifying requirements, DOD should consider providing the COCOMs with additional resources to establish robust analytical capabilities for identifying and assessing their capability needs. In addition, DOD and the Joint Staff should ensure 

the JROC gives equal consideration to COCOM needs. 

Dr. GANSLER. As Secretary Gates has stressed, we need a better ‘‘balance’’ between likely warfighter needs and resource allocations. Today, the budgets are unbalanced in the direction of potential, future peer competitors. 

Dr. KAMINSKI. The success is mixed. Warfighters are often very able to address 

their immediate needs, but addressing future needs can be challenging. Warfighters 

also need to be open to new conops—new approaches to performing their mission 

that can be derived from new, enabling technologies. Similarly, developers and cost 

estimators are more able to address near-term estimates involving technologies that 

are mature and familiar, and less able to do so with future technologies that are 

less mature and less familiar. Matching needs and resources must be a continuing, 

iterative process. This process must have a solid foundation in systems engineering, 

and the systems engineering must in turn have some foundation in M&S which is 

validated by building and testing. Effective value proposition assessments will occur 

only with skilled, trained, and experienced personnel from requirements and acquisition communities working together within a systems engineering framework to refine needs and solutions. 

Mr. ADOLPH. I don’t have the background or experience to respond to this question. 

TECHNOLOGICAL CHANGE AND WEAPONS SYSTEMS

20. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, one of the challenges we face with new technology and the rapid advancement 

of technology is the desire to outfit our military men and women with the latest and 

most advanced equipment. While we all want to provide our military with the latest 

advances in technology, I worry that as technology changes we are in an ever state 

of trying to upgrade a system while it is in development. How can we best handle 

constant technology changes for weapons systems that take years to develop and deliver? 

186 

Mr. SULLIVAN. First, technology development must be placed on the critical path 

of any new acquisition program. Technology development requires trial and error, 

discovery, and invention, which should all be done in a S&T environment before 

technologies are brought onto programs. Second, GAO has found that successful 

commercial companies use an evolutionary or incremental approach that reduces development risks and enables quicker delivery of products to the customer. These 

commercial companies have implemented an evolutionary approach by establishing 

time-phased plans to develop a new product in increments (5 years or less) based 

on technologies and resources achievable now and later. They set requirements for 

products that require only technologies that they know will work, and wait to put 

more risky technologies on future increments once their risks have been diminished. 

An open architecture approach to development is also important so that product 

components, such as technologies, can be upgraded or added in subsequent product 

development increments, without having to do a major redesign of the product. 

Historically, DOD’s approach to developing new weapon systems often attempts 

to satisfy the full capability in a single step, ‘‘big-bang’’ approach regardless of the 

design complexity or the maturity of the technologies. Under this approach, the 

warfighter can wait over a decade to receive any new capabilities and by the time 

the capabilities are delivered, they may be out of date. Implementing knowledgebased evolutionary acquisition practices—where individual program increments are 

defined on the basis of mature technologies and a feasible design that incorporates 

an open architecture approach—would allow DOD to more effectively manage technology changes in acquisition programs. Shorter, more manageable development cycles would also provide the warfighter with useful technologies quicker and provide 

more confidence that technologies can be developed within program cost estimates. 

Dr. GANSLER. See my answer to question #13. 

Dr. KAMINSKI. I have always found that the mark of a good program manager is 

a large lower right hand desk drawer. In this drawer are placed all the ideas for 

incorporation of new technology advancements until the first version of the system 

is fielded. At that time the drawer is opened, and representatives from the requirements and acquisition communities meet and apply systems engineering tools and 

techniques to assess shortfalls in the fielded system, evolving needs, and evolving 

technologies to examine value propositions and decide what to incorporate in the 

next block of that fielded system. 

Mr. ADOLPH. With platforms and major systems being procured less frequently 

and taking many years in development, there is often a tendency to over-reach since 

another opportunity to provide advanced technology to the warfighter may not occur 

for many years. The first, and most, beneficial action would be to work toward 

shortening the acquisition cycle. For various reasons, the acquisition cycle has become overly extended, due in part, as discussed in my response to question #13, to 

initiating acquisition programs based on immature technologies. This trend must be 

reversed. Recommendations have been made repeatedly in numerous studies that 

would result in a shortened cycle. 

One approach to a shortened cycle for large programs would be to incorporate new 

technologies in ‘‘block upgrades.’’ This approach has been successful for many defense systems, sometimes as a result of necessity and sometimes as preplanned improvements. The latter approach has been implemented on many aircraft programs 

where planned multi-staged improvement programs are executed at regular intervals. Strategies such as spiral acquisition and pre-planned product improvement 

have been pursued with various degrees of success. If too many new technologies 

are incorporated into a system, the risk is compounded by the complex task of integrating advanced technologies with sometimes insufficient technical maturity. These 

risks may be mitigated through disciplined processes to properly evaluate technology maturity, larger investments in technical domain expertise as well as in systems engineering. More emphasis should also be placed on analysis of alternatives, 

to include an OSD-level review. Designs that support ‘‘plug and play,’’ e.g. standard 

interfaces are also a partial solution. Producing the technology and integrating it 

into the logistics system as well as training maintenance personnel, et cetera, can 

still be challenges if the technology has never been deployed to the warfighter. (See 

also my response to question #13.) 

ACQUISITION PERSONNEL

21. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, a consistent theme throughout the hearing was the insufficiency of the current 

acquisition workforce. What would your recommendation be for increasing the number of Active Duty billets specifically for acquisition? 

187 

Mr. SULLIVAN. Although we did not examine DOD’s decisionmaking process for 

using military versus civilian personnel, we recently reported that DOD lacks critical Department-wide information to determine the sufficiency of its acquisition 

workforce to meet its mission needs. We recommended that DOD identify the number and skill sets of its total acquisition workforce—including civilian, military, and 

contractor personnel—and conduct analyses using this information to inform acquisition workforce decisions regarding the appropriate mix of civilian, military, and 

contractor personnel. We are encouraged by the Department’s recent announcement 

that it plans to significantly increase the size of DOD’s acquisition workforce by converting 11,000 contractors and hiring an additional 9,000 government acquisition 

professionals by 2015—beginning with 4,100 in fiscal year 2010. 

Dr. GANSLER. In the Commission I chaired (the ‘‘Commission on Army Acquisition 

and Program Management in Expeditionary Operations’’) we stressed that it is not 

just numbers that matter, but rank as well as experience. The senior military acquisition positions should be staffed with General Officers that have extensive acquisition experience. As to numbers, we should at least get back the 25 percent that Congress mandated be cut in fiscal year 1996 (which, across the DOD acquisition workforce, military and civilian, is approximately 50,000 additions—of which perhaps 20 

percent are military). 

Dr. KAMINSKI. Numbers alone aren’t the answer. We need an appropriate mix of 

quality and quantity. We need personnel able to provide early and continuing systems engineering and closely-coupled development planning. We need: a) sufficient 

personnel (in both government and industry) with adequate education, training and 

domain experience (this includes personnel in requirements development as well as 

in acquisition); and b) sufficient front end investment in the tools necessary to understand the key tradeoffs in cost/schedule/performance, and to identify and address 

the key risks in a systematic manner. 

Mr. ADOLPH. As a minimum, the government acquisition workforce, which includes military and civilian personnel, should be increased to the levels that existed 

in the mid-1990s prior to the congressional and Service reductions. The military/civilian acquisition position mix should be left to the individual Services as each is 

procuring different commodities, and has a different philosophy regarding a career 

military acquisition workforce. As I noted in my opening statement, during a time 

of increased programmatic and technical complexity, there has been a loss of a large 

number of the most experienced management and technical personnel, without an 

adequate replacement pipeline. Solutions to the acquisition problems must begin 

with reconstituting a trained and experienced government workforce, which includes 

programs managers, domain subject matter experts as well as systems engineers, 

contracts personnel, testers and evaluators. 

22. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, to ensure we get the quality and expertise we require, how do you recommend 

we recruit and retain this force? 

Mr. SULLIVAN. DOD has just announced plans to substantially increase its inhouse acquisition workforce over the next few years. Our recent report on DOD’s 

acquisition workforce included practices of leading companies that could provide insights for DOD as it moves forward to hire and retain these employees. We found, 

for example, that the companies employed a variety of recruitment and retention 

initiatives. More importantly, some companies assessed their efforts at filling workforce gaps by tracking data on specific recruiting and retention metrics. One company assessed the quality of their hiring sources by assessing the performance of 

new hires over their first 2 years. 

Dr. GANSLER. Promotion potential and acquisition-experienced leaders are the 

keys to recruiting, developing, and retaining top acquisition personnel. 

Dr. KAMINSKI. You are quite right in noting that we must address retention as 

well as recruitment to build the requisite work force. We want people who want to 

make a difference. When we recruit these people, we must expect that we won’t 

keep them if they can’t see that they in fact are making a difference. Excessive (and 

growing) time from program initiation to fielding is a big problem for young people 

who want to see the impact of their work. As this time increases from a few years 

to 15 years or more, it undermines the entire acquisition process by causing key 

participants to ‘‘lose the recipe’’, and lose a sense of accountability as well as a sense 

of being able to make a difference. When new capabilities are developed and fielded 

in 5 years, engineers, managers, testers, cost analysts, et cetera, are able to benefit 

from and apply the experience gained from a previous program or program phase. 

They can also see the results of their decisions and be held accountable. We can 

also meaningfully employ past performance of the contractor as a factor in the 

award of future programs—an important factor in incentivizing contractor performVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00191 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
188 

ance. This all changes dramatically when the time extends to 15 years, and we have 

five roll-overs of management, engineers, cost analysts, and commercial technology 

during this time period. This long and growing time period is a result of the inflexibility inherent in our entire system of requirements development, budgeting, and acquisition, and it creates a vicious cycle in which it further exacerbates the contributors above, and they in turn further increase the time and cost growth. We see the 

result when we must discard our current acquisition system in order to deal with 

urgent needs and field systems such as MRAP and jammers to counter IEDs by 

forming and using rapid reaction organizations. This cycle must be broken by attacking the root causes. 

Mr. ADOLPH. When the acquisition workforce reductions were made in the late 

1990s, they were implemented by encouraging early retirements as well as hiring 

freezes, which often lasted for years. As a consequence of the latter, the pipeline 

of new civilian employees was shut off, and today there are large gaps in experience 

as well as inadequate numbers of personnel. Some Service test organizations are aggressively addressing these issues and have more insight into what works in today’s 

environment than I do. Based on my past experience, initial hiring, particularly for 

positions at remote test locations, is the biggest challenge. It requires aggressive recruiting at universities, along with co-op programs. The challenge of the work environment and the learning experiences, dealing with the latest technologies, combined with well-funded full-time advanced degree educational opportunities (with 

associated follow-on career commitments), are powerful positive retention forces 

after people are brought on board. On-the-job learning experiences have been diminished in some test organizations, where government personnel have been relegated 

to less challenging supporting functions. 

23. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, what can we do in the meantime to handle the personnel shortage since it will 

take many years to develop the level of expertise required for proper acquisition reform? 

Mr. SULLIVAN. As we recently reported in our review of DOD’s acquisition workforce management, DOD is currently implementing initiatives aimed at improving 

the capacity of its acquisition workforce using the newly established Acquisition 

Workforce Development Fund. These efforts are focused in three key areas: (1) recruiting and retention; (2) training and development; and (3) retention and recognition. DOD has also recently announced plans for significantly increasing its in-house 

acquisition workforce over the next few years. While DOD integrates these additional employees into its acquisition workforce, the Department can also leverage 

the experience and knowledge of its current workforce. DOD can take steps to retain 

experienced personnel to aide in the development of newer staff and to ensure it 

maintains and transfers key institutional knowledge. 

Dr. GANSLER. To fill the gap, while developing a new group, I would bring into 

the government some middle and senior people from industry; and I would use contractors (with experience, but without conflicts) to fill the rest of the slots. 

Dr. KAMINSKI. We can begin to hire people with education in engineering and systems engineering. We can send our current personnel back to universities to enhance their education. We can send our current personnel to industry to gain domain experience. We can bring back retirees on a part time basis to provide mentoring to our new hires and help them gain domain experience. We need to do all 

of these. 

Mr. ADOLPH. During the period when the normal pipeline is being replenished, 

Centers of Excellence must be created in selected test and acquisition engineering 

support organizations, with a focus on technical domain subject matter expertise. 

These cadres of expertise will provide the basic oversight functions such as those 

I outlined in my answer to question #16. The first step is to provide oversight in 

key technical disciplines at key programmatic times, e.g., RFP preparation, source 

selection, TEMP preparation, test reporting to support program reviews; 

transitioning to more active, continuous involvement as the workforce increases. 

The Services may be reluctant to take this approach. These Service cadres can also 

be augmented by subject matter experts from FFRDCs. The use of red teaming by 

outside experts (e.g., cross-Service, FFRDCs, and SETAs) should also be employed 

to provide the requisite expertise, as well as a measure of independence which is 

too often lacking in government program office-centric reviews. 

189 

MANDATORY PROCEDURES FOR MAJOR DEFENSE ACQUISITION PROGRAMS

24. Senator BEN NELSON. Mr. Sullivan, Dr. Gansler, Dr. Kaminski, and Mr. Adolph, what oversight and reporting mechanisms do you recommend we establish to 

ensure the Mandatory Procedures for Major Defense Acquisition Programs (MDAPs) 

meet warfighter requirements while eliminating redundancy and wasteful spending? 

Mr. SULLIVAN. DOD has recently revised its acquisition policy in ways intended 

to provide key Department leaders with the knowledge needed to make informed decisions before a program starts and to maintain disciplined development once it begins. The revised policy recommends the completion of key systems engineering activities before the start of development, requires early prototyping, establishes review boards to identify and mitigate technical risks, and implements early milestone 

reviews. If implemented, these policy changes could help programs replace risk with 

knowledge, thereby increasing the chances of developing weapon systems within 

cost and schedule targets while meeting user needs. To achieve improved outcomes 

in acquisition programs, DOD must ensure its policy changes are consistently implemented and reflected in decisions on individual programs. However, DOD could do 

more by requiring new programs to have manageable development cycles and establish knowledge-based cost and schedule estimates. Limiting the length of development cycles would make it easier to more accurately estimate costs, predict future 

funding needs, effectively allocate resources, and hold decision makers accountable. 

The acquisition reform legislation recently proposed by the Senate Armed Services 

Committee should also help achieve further improvements. Provisions increasing 

the emphasis on systems engineering, requiring early preliminary design reviews, 

and strengthening independent cost estimates and technology readiness assessments should make the critical front end of the acquisition process more disciplined. 

Establishing a termination criterion for critical cost breaches could help prevent the 

acceptance of unrealistic cost estimates at program initiation. In addition, having 

greater COCOM involvement in determining requirements and greater consultation 

between the requirements, budget, and acquisition processes could help improve 

DOD’s efforts to balance its portfolio of weapon system programs. 

While policy revisions and legislation may lead to improvements, they will not be 

effective without changes to DOD’s overall acquisition environment and the incentives that drive the behavior of its decisionmakers, the military Services, program 

managers, and the defense industry. Changing the environment will require strong 

leadership and accountability within the Department. 

Dr. GANSLER. I believe we currently have enough oversight and reporting mechanisms for MDAPs (and more would only slow the system down even more). The critical need is to use these mechanisms more effectively—with experienced personnel 

(who know what to look for, and what questions to ask). 

Dr. KAMINSKI. We first need to realize that ‘‘one size does not fit all.’’ For example, programs dealing with countering IEDs need cycle times measured in weeks, 

while a next generation bomber program must plan for fielding and support cycles 

measured in years. 

In most cases today the program manager’s authority is diffused by many levels 

of oversight in both the Department and in Congress, and the financial and performance constraints imposed do not allow sufficient freedom of action to apply informed judgment in a timely manner. Flexibility is further limited by application 

of a ‘‘one-size-fits-all’’ approach imposed by the DOD 5000 system, and the oversight 

practiced by the DOD and Congress. A program manager needs the freedom to tailor the acquisition approach to the problem, to ensure that the program response 

time will fall within the response time of the threat, and to apply a variety of tools 

and techniques (such as the use of prototypes, competitive prototypes, M&S, critical 

subsystem and component demonstration). For this to work, we need program managers with the education, training, and domain experience needed to enable timely 

responses and excellent judgment relevant to the domain. They need the flexibility 

to apply their expertise. 

Mr. ADOLPH. I believe the current oversight and reporting mechanisms, when 

combined with the changes in the proposed legislation, will provide adequate direction and guidance for the acquisition process. The key is putting more discipline in 

every step of the process by the Services and OSD. This was achieved in numerous 

past successful programs in the 1960s, 1970s, and 1980s without being overly bureaucratic. One impediment is the recent trend towards contractual vehicles which 

relieve development contractors of reporting requirements, including regular status 

reports on costs and technical performance. The government must have continued 

access to cost and performance data to provide effective oversight. 

Most of the basic policies and directives for the acquisition process represent best 

practices based on decades of experience. The fundamental problem is a lack of adeVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00193 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
190 

quate discipline throughout the process. This begins with proper definition of operational requirements at the outset of a program and stating those requirements in 

terms that are reasonable and quantifiable for design and test purposes. Another 

major issue is the lack of adequate technology maturity (See my response to question #13.) 

There have been some problems with military specifications and standards in the 

past, in that they were not properly tailored to the system to be procured. However, 

these standards have evolved and were updated over decades and were, for the most 

part, excellent compendiums of best practices and lessons learned. If properly used, 

they provide guidance for system development to ensure a systematic and disciplined approach. Proper application of these guidance documents also ensures that 

problems, which are a normal part of any complex high technology development, are 

identified early. Many of these documents have been allowed to atrophy over the 

last several years or have been eliminated. Previously eliminated specifications and 

standards should be selectively updated and reinstated; while retaining the option 

to use commercial specifications and standards when available, provided the latter 

adequately address the requisite military performance and suitability requirements 

in the intended operating environment. Finally, the most important ingredient remains a capable and experienced government acquisition workforce with equally capable leaders, who remain on the job long enough to realize the consequences of 

their programmatic decisions. 

QUESTIONS SUBMITTED BY SENATOR DAVID VITTER

PERSONNEL IN ACQUISITION REFORM

25. Senator VITTER. Dr. Gansler, a recurring message from the panel is the importance of improving the quality and experience of personnel in acquisition reform, 

and I’m glad that Chairman Levin and Ranking Member McCain included section 

206 in their acquisition reform bill to reward excellence. This section requires the 

establishment of a program recognizing excellent performance and the award of 

cash bonuses. However, it was stated that cash isn’t necessarily the greatest incentive for encouraging excellence, and making a difference is the best incentive. In 

your opinion, what can DOD implement in an excellence program to demonstrate 

improvements so that personnel can track and actually see their work reflected in 

tangible results to provide and maintain motivation? 

Dr. GANSLER. Combining appropriate metrics with rewards for success (including 

recognition, and even token financial rewards) are desirable—but the managers 

must be allowed to have sufficient flexibility to exercise their judgment, if they are 

to succeed. 

26. Senator VITTER. Dr. Gansler, how much of the problem in attracting the right 

quantity of qualified acquisition personnel is a product of the overall Federal employee hiring rules established by Office of Personnel Management (OPM) or the 

specific rules established by DOD? 

Dr. GANSLER. The DOD must be able to compete with industry for ‘‘the best and 

brightest.’’ The process must be flexible and fast—yet fair. Salary will not be the 

driver (government jobs offer the rewards) but the process can be neither bureaucratic nor political. 

27. Senator VITTER. Dr. Gansler, last year, Congress authorized $300 million to 

rebuild the acquisition workforce. Do you believe this is sufficient? 

Dr. GANSLER. It is an important start, and will likely need to be continued for 

a few years. 

28. Senator VITTER. Dr. Gansler, cost overruns were endemic in the 1980s and 

1990s when the acquisition workforce was robust. Is it likely then that adding more 

people will solve the problem of cost overruns? 

Dr. GANSLER. As I noted in question #21, numbers alone are not the answer, senior and experienced people with an improved acquisition process (as covered in my 

answers above—especially numbers 1, 8, and 13). 

29. Senator VITTER. Dr. Gansler, it seems that there is a problem with keeping 

quality program managers in their roles for an extended period of time. How can 

we ensure continuity at these positions? 

Dr. GANSLER. Goldwater-Nichols intended the program managers to remain at 

least through the next milestone. In the Packard Commission we emphasized that 

191 

they be allowed to be promoted in place (so that the extended period is a reward, 

not a career-limiting assignment). 

FUNDING FOR DEVELOPMENTAL TESTING

30. Senator VITTER. Mr. Adolph, to what extent does a lack of funding for developmental testing (DT) early in a program’s schedule result in cost increases and delays 

later in its schedule? 

Mr. ADOLPH. The lack of adequate funding for DT has a major adverse impact 

on developmental cost and schedule. There are numerous issues relating to DT 

funding. The weapons timelines for RDT&E continue to increase, driven by inadequate resourcing of the entire developmental process as well as the inclusion of immature technologies in systems. Inadequate funding of DT, which includes the number of test assets, results in delayed identification and correction of problems, many 

of which are a normal part of a high-technology developmental program. Late identification results in more difficulty and expense involved in fixes. Most recent programs have had an inadequate number of assets to execute a robust DT program 

in a timely manner. When there are an inadequate number of test assets, the DT 

program is drastically impacted, often for months, when a test vehicle must be laid 

up for a retrofit. By the time a program reaches the full-scale platform test phase, 

there is a huge cadre of test and test support personnel who cannot be efficiently 

used whenever a test program comes to a halt. The fixed cost of maintaining this 

cadre often exceeds the incremental variable costs of conducting test missions. Another factor in efficient testing is the availability of adequate test personnel and test 

facilities. The latter includes physical test support facilities and assets, instrumentation, and data processing (see also my response to question #17). Numerous developmental programs have been delayed because of the lack of adequate facilities, capacity, and people to accomplish the basic data processing tasks quickly, as well as inadequate number of domain subject matter experts to analyze test results. This is 

exacerbated when earlier involvement by DT&E and OT&E personnel is limited by 

a lack of sufficient experienced personnel. Commercial programs routinely resource 

facilities and personnel to conduct testing on a multiple shift basis during critical 

phases of the developmental process. 

COMPETITIVE PROTOTYPING

31. Senator VITTER. Dr. Kaminski, how much can competitive prototyping help in 

identifying the best product for warfighters? 

Dr. KAMINSKI. Competitive prototyping is one arrow in our quiver of tools and 

techniques in development planning. Not all prototyping needs to be competitive. 

Prototyping should not be limited to the full system level. There are cases where 

prototyping is best applied only to address major platform integration risk areas, 

or to critical subsystems. Again, I don’t believe there is a ‘‘one-size-fits-all’’ solution 

to prototyping. But I do believe that at a big picture level we are not doing enough 

prototyping in our development planning. Reducing risk and developing domain expertise require that we build things. M&S are of great value, and so are our computer based analysis and design tools, but all of my experience tells me that we 

need to build and test on a continuing basis to make mistakes, learn from those 

mistakes, and apply that learning in the development of new capabilities. 

32. Senator VITTER. Dr. Kaminski, the F–16 program, lauded by many as an example of how to acquire a capability, was competed to the point where the Air Force 

actually flew competing planes before deciding on which defense contractor won the 

award. Why have we gotten so far away from that? 

Dr. KAMINSKI. We haven’t completely departed from that approach. We did something very similar in the JSF program. But I agree that we are not doing enough. 

We can do quite a bit more. I believe we can do so without increasing costs by 

spending money upfront to reduce or avoid the cost of significant overruns later. 

JOINT REQUIREMENTS OVERSIGHT COUNCIL

33. Senator VITTER. Mr. Sullivan, the Vice Chiefs have important jobs internally 

managing their respective Services. Do you believe that JROC should instead be 

staffed with independent personnel who are not advocates for a particular Service? 

Mr. SULLIVAN. GAO has found DOD’s requirements process to be too Service-centric. The needs of the COCOMs should be given greater consideration. There are a 

number of ways the requirements setting process might be changed to address this 

192 

issue. For example, the combatant commanders might supplant the Service Vice 

Chiefs role in the process or might be given an equal say in the process. Several 

outside reviews have suggested such solutions: 

• The Beyond Goldwater-Nichols study team recommended a more ‘‘joint’’ 

JROC in which the Service Vices are replaced by COCOM Deputies and civilians responsible for requirements policy. 

• The Defense Acquisition Performance Panel (DAPA) recommended replacing the current requirements setting process with a combatant commanderled requirements process in which the Services and defense agencies compete to provide solutions. Each of the combatant commanders would be 

tasked to prepare extended planning annexes to each of their operational 

and contingency plans (to be updated on a 2-year cycle) that would provide 

a 15-year forecast of both capability gaps and excesses relative to mission 

requirements. 

• Some members of the Defense Science Board Summer Study on Transformation felt that the requirements process continues to be dominated by 

the force providers and the Joint Staff and that COCOM needs are underrepresented. They proposed making the COCOMs more equal partners with 

the force providers from the beginning of the process, particularly when 

identifying capabilities needed to carry out the Department’s operational 

missions. 

We believe that these proposals deserve further consideration as a means to help 

improve the Department’s ability to balance joint warfighting needs. 

34. Senator VITTER. Mr. Sullivan, should we have a new, independent JROC that 

could hear all advocates for all programs and then recommend continuation or elimination of the program based on current and likely future threats to the United 

States? 

Mr. SULLIVAN. Because many major defense acquisition programs have ambitious 

and lengthy product developments—in some cases, delivery of a weapon system to 

the warfighter takes as long as 15 years—there can be a significant amount of time 

that elapses between JROC reviews and changes in the threat environment can 

occur. Before a major weapon program is approved to begin system development, the 

JROC must validate the capabilities to be developed, including the system’s key performance parameters. The JROC also validates the production elements of a program before it is approved to start production. For programs that have lengthy development cycles, more frequent reviews may be necessary to assess whether programs are worth continuing in light of current and likely future threats. In prior 

work we have conducted on how successful companies manage their portfolios of 

product development efforts, we found that the companies revisit their investment 

decisions at multiple stages throughout product development to ensure products are 

still of high value. 

35. Senator VITTER. Mr. Sullivan, should the Services be required to budget their 

programs to the cost estimates provided by a new Director of Independent Cost Assessment? 

Mr. SULLIVAN. Budgeting programs to an independent cost estimate would be an 

improvement, however, for cost estimates to be effective they must be based on a 

high degree of knowledge about requirements, technology, design, and manufacturing. 

A reliable cost estimate helps ensure a program’s projected funding needs are adequate to execute the program. Less than a quarter of the 48 programs in GAO’s annual assessment of weapon system programs that provided data used the estimate 

made by DOD’s Cost Analysis Improvement Group (CAIG) as a basis for the program’s baseline, while almost 70 percent of the programs used the program office 

or Service cost estimate. While cost estimates from the CAIG can underestimate a 

program’s costs by billions of dollars, GAO has previously found that these independent estimates generally underestimate costs by a smaller amount than program 

office and Service estimates. 

Cost estimates are inaccurate in part because they are based on limited knowledge about the requirements, technologies, design maturity, and the time and funding needed to execute a program. GAO has found, for example, that program Cost 

Analysis Requirements Documents—used to build the program cost estimate—often 

lack sufficient detail about planned program content for developing sound cost estimates. Without this knowledge, cost estimators must rely heavily on parametric 

analysis and assumptions about system requirements, technology, and design maturity, and the time and funding needed. The assumptions used in developing estiVerDate Nov 24 2008 13:52 Nov 09, 2009 Jkt 000000 PO 00000 Frm 00196 Fmt 6633 Sfmt 6621 S:\WPSHR\BORAWSKI\DOCS\53267.TXT JUNE PsN: JUNEB
193 

mates also tend to be overly optimistic. Furthermore, cost estimates that lack 

knowledge and rely on assumptions have inherently high levels of risk and uncertainty that are not typically communicated to decision makers. 

36. Senator VITTER. Mr. Sullivan, with the annual Selected Acquisition Reports 

and current Nunn-McCurdy reporting requirements, is Congress suffering from a 

lack of oversight and reports from DOD? 

Mr. SULLIVAN. In terms of the reports it receives from DOD, Congress is suffering 

from a lack of insight into the risks on programs that could lead to cost, schedule, 

and performance shortfalls. The Selected Acquisition Reports and Nunn-McCurdy 

reporting that Congress gets from DOD on its major defense acquisition programs 

primarily report on program outcomes to date—including quantitative measures of 

cost, schedule, and performance over time. While these are useful indicators of the 

health of acquisition programs and whether they are meeting their intended goals, 

they have limited predictive value. DOD could improve the information it provides 

to Congress by reporting on knowledge metrics that evaluate whether programs 

have attained certain levels of technology, design, and manufacturing knowledge by 

key points in the acquisition process. This reporting would facilitate the identification of potential problems that could lead to cost, schedule, and performance shortfalls before they occur. 

37. Senator VITTER. Mr. Sullivan, is there any credence to the concern that program managers have ‘‘too many’’ reports to file and are not focusing on managing 

their programs? 

Mr. SULLIVAN. Oversight and reporting requirements need to be evaluated both 

from the perspective of the time they take and the value they provide. In the past, 

we have reported on concerns in both of those areas. In a November 2005 report, 

we reported that program managers and program executive officers commented that 

they spend too much time producing data that is not strategic or very useful to 

them. In fact, more than 90 percent of 126 respondents to our survey of program 

managers said that they spent either a moderate, great, or very great extent of their 

time representing their program to outsiders and developing and generating information about program progress. 

We have also found that the reporting that is done does not include key information that decisionmakers need to decide if a program is ready to proceed into the 

next phase of the acquisition process. For example, in transitioning from system integration to system demonstration, we have recommended that DOD ensure the capture of knowledge about the completion of engineering drawings; completion of subsystem and system design reviews; agreement from all stakeholders that the drawings are complete; and identification of critical manufacturing processes, among 

other indicators. In the transition to production, we recommended that DOD capture 

knowledge about production and availability of representative prototypes along with 

statistical process control data. Our 2005 report indicated that a relatively small 

percentage of programs used these knowledge indicators. For example, only 32 percent of 126 program managers who responded to our survey said they used design 

drawing completion extensively to measure design maturity; only 26 percent said 

they used production process controls to a great extent. Even fewer program managers reported that their immediate supervisor used these measures extensively to 

evaluate progress. Broader use and reporting of these types of metrics would help 

decisionmakers gauge progress and ensure that programs managers are: (1) conducting activities to capture relevant product development knowledge; (2) providing 

evidence that this knowledge has been captured; and (3) holding decision reviews 

to determine that the requisite knowledge has been captured before proceeding to 

the next phase of the acquisition process. 

[Whereupon, at 11:52 a.m., the committee adjourned.] 

Æ