1 GAO-13-826R VXX Prototyping Waiver

441 G St. N.W.

Washington, DC 20548

September 6, 2013

Congressional Committees

Department of Defense’s Waiver of Competitive Prototyping Requirement for the VXX 

Presidential Helicopter Replacement Program

The Weapon Systems Acquisition Reform Act of 2009, as amended (WSARA), requires the 

Secretary of Defense to modify guidance to ensure that the acquisition strategy for each major 

defense acquisition program provides for competitive prototypes before Milestone B approval—

which authorizes entry into system development—unless the Milestone Decision Authority 

waives the requirement.1 Competitive prototyping, which involves commercial, government, or 

academic sources producing early prototypes of weapon systems or critical subsystems, can 

help Department of Defense (DOD) programs reduce technical risk, refine requirements, 

validate designs and cost estimates, and evaluate manufacturing processes prior to making 

major commitments of resources. It can also help reduce the time it takes to field a system, and 

as a result, reduce its acquisition cost. WSARA states that the Milestone Decision Authority may 

waive the competitive prototyping requirement only on the basis that (1) the cost of producing 

competitive prototypes exceeds the expected life-cycle benefits (in constant dollars) of 

producing such prototypes, including the benefits of improved performance and increased 

technological and design maturity that may be achieved through competitive prototyping; or (2)

but for such a waiver, DOD would be unable to meet critical national security objectives.

WSARA also provides that whenever a Milestone Decision Authority authorizes a waiver of the 

competitive prototyping requirement on the basis of what WSARA describes as “excessive 

cost,” the Milestone Decision Authority is required to submit notification of the waiver, together 

with the rationale, to the Comptroller General of the United States at the same time it is 

submitted to the congressional defense committees. WSARA further provides that no later than 

60 days after receipt of a notification of a waiver, we are mandated to review the rationale for 

the waiver and submit a written assessment of that rationale to the congressional defense 

committees.

2

On July 10, 2013, we received notice from DOD that it had waived the competitive prototyping 

requirement for the VXX Presidential Helicopter Replacement Program. The Navy's VXX 

 1

Pub. L. No. 111-23, § 203(a), as amended by the Ike Skelton National Defense Authorization Act for Fiscal Year 

2011, Pub. L. No. 111-383, § 813. DOD modified its guidance related to the operation of its acquisition system 

through Directive-Type Memorandum (DTM) 09-027, “Implementation of Weapon System Acquisition Reform Act of 

2009 (Dec. 4, 2009, incorporating Change 3, Dec. 9, 2011). Major defense acquisition programs are those estimated 

by DOD to require an eventual total expenditure for research, development, test, and evaluation of more than $365 

million, or for procurement, including all planned increments or spirals, of more than $2.19 billion in fiscal year 2000 

constant dollars. The Milestone Decision Authority for major defense acquisition programs is the Under Secretary of 

Defense for Acquisition, Technology, and Logistics; the head of a DOD component; or if delegated the component 

acquisition executive.

2

Pub. L. No. 111-23 § 203(b)(1) & (2).
2 GAO-13-826R VXX Prototyping Waiver

program is an effort to replace the aging fleet of 19 VH-3D and VH-60N helicopters used to 

transport the President, Vice President, heads of state, and others. The program is the Navy’s 

second effort to replace these helicopters. The first effort, the VH-71 program, was canceled in 

2009 due to significant cost growth, schedule delays, and performance issues. Since then, the 

Navy has reduced the helicopter’s passenger capacity, range, transportability, operational 

availability, airspeed, and hover performance requirements. These performance trade-offs 

allowed the Navy to adopt a streamlined acquisition approach that involves integrating mature 

communications and mission systems into an existing commercial or military helicopter that is 

already in production without the need for a technology development phase. The Navy expects 

this approach to lower the program’s cost, shorten its schedule, and reduce its risk.

In this report, we assess DOD’s rationale for waiving the competitive prototyping requirement for 

the VXX program and the analysis used to support it. To conduct our assessment, we compared 

the rationale in the waiver to the WSARA requirement to determine the extent to which the 

waiver is consistent with the statute. In addition, we reviewed the Navy’s cost-benefit analysis, 

which provides the data and assumptions on which the waiver is based. We also questioned

DOD, Navy, and VXX program officials to clarify information in this documentation, as 

necessary. Further, we relied on our April 2013 report on the VXX program for information on 

the degree to which the cost estimates used to support the Navy’s cost-benefit analysis 

complied with cost estimating and assessment best practices.3 We did not assess the estimated 

cost benefits of the various prototyping scenarios in the Navy’s cost-benefit analysis against 

these best practices criteria, but did take other steps to assess their reasonableness.

We conducted this performance audit from July 2013 to September 2013 in accordance with 

generally accepted government auditing standards. Those standards require that we plan and 

perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on our audit objectives.

Results in Brief

DOD’s rationale for waiving WSARA’s competitive prototyping requirement for VXX addresses 

one of the two bases provided in the statute; namely that the cost of producing competitive 

prototypes exceeds the expected life-cycle benefits (in constant dollars) of producing the 

prototypes. The VXX program’s acquisition strategy provided the primary justification for the 

prototyping waiver. According to the waiver, VXX requirements can be met by integrating an 

existing, in-production, flight-proven aircraft with mature mission systems. The Navy in its waiver 

request also concluded that the integration activities planned for the VXX program do not 

require additional technology maturation or risk reduction beyond that already being 

accomplished by the government through its own prototyping of certain critical mission 

subsystems. Recognizing that the intent of competitive prototyping is to reduce cost and risk, 

DOD has taken other actions that could arguably achieve these goals. Specifically, DOD 

decided to reduce requirements, use an existing aircraft, and mature critical subsystems before 

integrating them on the aircraft. In the waiver, DOD also found reasonable the Navy’s costbenefit analysis, which examined multiple acquisition strategies with system- and subsystemlevel prototyping from one or two contractors. In all, the Navy examined six different acquisition 

 3

GAO, Presidential Helicopter Acquisition: Program Makes Progress in Balancing Requirements, Costs, and 

Schedule, GAO-13-257 (Washington, D.C.: Apr. 9, 2013).
3 GAO-13-826R VXX Prototyping Waiver

strategies and concluded that requiring competitive prototyping would delay fielding an initial 

operational capability by 16 months and increase development costs by about $782 million to 

$3.38 billion (in base year 2011 dollars), depending on the type and number of prototypes. The 

Navy also estimated that the more costly system-level prototyping strategies could achieve an 

estimated $542 million (in base year 2011 dollars) in life-cycle cost benefits by improving the 

reliability of the aircraft, which in turn could reduce the number of helicopters required. 

According to the Navy, the cost data used in its cost-benefit analysis were drawn from the VXX 

analysis of alternatives study and related activities. In April 2013, we found that the Navy used 

cost estimating procedures on the VXX analysis of alternatives that substantially complied with

best practices.

Waiver Rationale Is Consistent with WSARA and Supporting Analysis Incorporated 

Lessons Learned from Prior Waivers

DOD’s rationale for waiving WSARA’s competitive prototyping requirement for VXX addressed 

one of the two bases provided for a waiver in the statute, namely that the cost of producing 

competitive prototypes exceeds the expected life-cycle benefits, including the benefits of 

improved performance and increased technological and design maturity that may be achieved 

through competitive prototyping. In the waiver, DOD also stated that the cost of producing a 

single prototype of the VXX or its critical subsystems before Milestone B exceeded the expected 

life-cycle benefits.4 The VXX program’s acquisition strategy provided the primary justification for 

the prototyping waiver. According to the waiver, VXX requirements can be met by integrating an 

existing, in-production, flight-proven aircraft with mature mission systems. The program is 

conducting government-led prototyping of certain critical mission subsystems, including an 

Internet Protocol-based communication capability that is comprised of existing commercial 

equipment. We have previously found that there are potential risks associated with integrating 

government-developed mission systems into a new airframe.5 The Navy’s waiver request 

acknowledged that integration could be a challenge, but it concluded that the integration 

activities planned for the VXX program do not require technology maturation or risk reduction 

beyond that already being accomplished by the government. Therefore, it concluded that the 

program would not benefit from additional prototyping at the system or subsystem level.

Recognizing that the intent of competitive prototyping is to reduce cost and risk, DOD has taken

other actions that could arguably achieve these goals. Specifically, DOD decided to reduce

requirements, use an existing aircraft, and mature critical subsystems before integrating them 

on the aircraft.

In the waiver, DOD found reasonable the Navy’s cost-benefit analysis, which examined multiple 

acquisition strategies with system- and subsystem-level prototyping from one or two contractors. 

The Navy’s approach was consistent with a key principle in DOD’s policy on economic analysis, 

which states that each feasible alternative for meeting an objective must be considered, and its 

life-cycle costs and benefits evaluated.6

 4

WSARA provides that whenever a Milestone Decision Authority authorizes a competitive prototyping waiver, the 

program is required to produce a prototype prior to Milestone B approval if the expected life-cycle benefits (in 

constant dollars) of producing such prototype exceed its cost and its production is consistent with achieving critical 

national security objectives. Pub. L. No. 111-23 § 203(a)(3)(A).

In all, the Navy examined six different acquisition 

5

GAO-13-257.

6

DOD Instruction 7041.3, Economic Analysis for Decisionmaking (Nov. 7, 1995).
4 GAO-13-826R VXX Prototyping Waiver

strategies with various degrees of prototyping and compared the development cost of the VXX 

as well as the potential life-cycle benefits of prototyping under each scenario. Based on this 

analysis, DOD concluded that requiring competitive prototyping of critical subsystems would 

delay fielding an initial operational capability by 16 months and increase development costs by 

about $1.189 billion for two industry prototypes and $782 million for a single prototype in base 

year 2011 dollars. The waiver also concluded that pursuing an aircraft prototyping strategy to 

improve aircraft performance would increase development costs by about $3.38 billion for a 

strategy with two industry prototypes and $2.84 billion for a single prototype in base year 2011 

dollars. Table 1 includes a description of each of these acquisition strategies and their total 

estimated development cost, schedule, and expected benefits.

Table 1: Navy’s Estimated Cost, Schedule, and Benefits for Various VXX Acquisition Strategies

Acquisition strategy Total development cost 

including prototyping (in 

base year 2011 billions)

Time to field initial 

capability (in months)

Expected benefits of 

prototyping

Planned strategy with 

government prototyping of 

critical mission subsystems 

and no aircraft prototypinga

$4.33-4.51 85

Subsystem prototypes

One contractor prototyping 

critical mission subsystems

5.29 101 None

Two contractors prototyping 

critical mission subsystems

5.70 101 None

Aircraft prototypesb

Major development program 

with one aircraft prototype

7.17 At least 101 Life cycle cost savings of 

$542 million (base year 

2011 dollars) driven by 

improved operational 

availability and reduction in 

number of helicopters 

required; potential capability 

improvements

Major development program 

with two competing aircraft 

prototypes

7.71 At least 101 Life cycle cost savings of 

$542 million (base year 

2011 dollars) driven by 

improved operational 

availability and reduction in 

number of helicopters 

required; potential capability 

improvements

Source: GAO analysis of Navy’s VXX cost-benefit analysis. a

This strategy was treated as two separate scenarios in the Navy’s cost-benefit analysis.

b

Assumes government prototyping of critical mission subsystems.

The Navy used cost estimating procedures on the VXX competitive prototyping cost-benefit 

analysis that were substantially compliant with best practices.7

 7

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program 

Costs, 

According to the Navy, the cost 

data used in its cost-benefit analysis were drawn from the VXX analysis of alternatives study 

and related activities. This analysis was completed from 2010 to 2012 and provided the basis 

for the program’s current acquisition strategy. In our April 2013 report, we found that the cost 

GAO-09-3SP (Washington, D.C.: Mar. 2, 2009).
5 GAO-13-826R VXX Prototyping Waiver

estimating procedure used for the analysis of alternatives was substantially compliant with best 

practices criteria.

8 Specifically, we found the analysis of alternatives cost estimate to be 

comprehensive and well documented. We also found that it was substantially accurate and 

partially met our criteria for being credible. The cost estimating best practices associated with 

each of those characteristics used in assessing the analysis of alternatives are provided in 

enclosure 1.

The Navy incorporated lessons learned from DOD’s first two competitive waivers to improve 

certain aspects of the VXX cost-benefit analysis. In an August 2012 report, we concluded that

the rationale for the Enhanced Polar System program waiver was consistent with WSARA, but 

the cost-benefit analysis supporting it was incomplete.9 That analysis did not include a specific 

dollar value or range of values for the estimated life-cycle benefits of competitive prototyping. 

We also found that the Air Force’s estimate of the cost of competitive prototyping was prepared 

by the program office and not independently estimated or reviewed by other DOD cost 

estimating organizations. In a March 2013 report, we found that the Air Force only evaluated 

one potential approach to implementing competitive prototyping in its waiver request for the 

Combat Rescue Helicopter program.10 We concluded that the Air Force could have more fully 

evaluated the cost and benefits of additional prototyping strategies, including more limited 

approaches that are mentioned in WSARA or DOD’s implementing memorandum, such as 

producing a single prototype or prototyping critical subsystems before Milestone B. The VXX

cost-benefit analysis addressed all of these points.

 8

GAO-13-257.

9

GAO, Department of Defense’s Waiver of Competitive Prototyping Requirements for Enhanced Polar System 

Program, GAO-12-983R (Washington, D.C.: Aug. 23, 2012).

10GAO, Department of Defense’s Waiver of Competitive Prototyping Requirements for Combat Rescue Helicopter

Program, GAO-13-313R (Washington, D.C.: Mar. 7, 2013).
6 GAO-13-826R VXX Prototyping Waiver

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. In its written comments, which are 

reprinted in enclosure 2, DOD concurred with our assessment of the VXX prototyping waiver.

- - - - - -

We are sending copies of this report to interested congressional committees, the Secretary of 

Defense, the Secretary of the Navy, and the Commandant of the Marine Corps. In addition, the 

report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions, please contact me at (202) 512-4841 or 

sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs 

may be found on the last page of this report. GAO staff who made key contributions to this 

report were Ron Schwenn, Assistant Director; Bruce Thomas, Assistant Director; Marie P. 

Ahearn; Pedro Almoguera; Jerry Clark; Bonita Oden; Kenneth E. Patton; and Carol Petersen.

Michael J. Sullivan

Director, Acquisition and Sourcing Management

Enclosures - 2
7 GAO-13-826R VXX Prototyping Waiver

List of Committees

The Honorable Carl Levin

Chairman

The Honorable James Inhofe

Ranking Member

Committee on Armed Services

United States Senate

The Honorable Richard J. Durbin

Chairman

The Honorable Thad Cochran

Ranking Member

Subcommittee on Defense

Committee on Appropriations

United States Senate

The Honorable Howard P. “Buck” McKeon

Chairman

The Honorable Adam Smith

Ranking Member

Committee on Armed Services

House of Representatives

The Honorable C.W. Bill Young

Chairman

The Honorable Pete Visclosky

Ranking Member

Subcommittee on Defense

Committee on Appropriations

House of Representatives
8 GAO-13-826R VXX Prototyping Waiver

Enclosure-1

Cost Estimating Best Practices

Characteristic Best practices

Comprehensive • The cost estimate includes all life cycle costs.

• The cost estimate completely defines the program, reflects the current schedule, and 

is technically reasonable.

• The cost estimate work breakdown structure (WBS) is product-oriented, traceable to 

the statement of work/objective, and at an appropriate level of detail to ensure that 

cost elements are neither omitted nor double-counted.

• The estimate documents all cost-influencing ground rules and assumptions. 

Well documented • The documentation should capture the source data used, the reliability of the data, 

and how the data were normalized.

• The documentation describes in sufficient detail the calculations performed and the 

estimating methodology used to derive each element’s cost.

• The documentation describes step by step how the estimate was developed so that a 

cost analyst unfamiliar with the program could understand what was done and 

replicate it.

• The documentation discusses the technical baseline description and the data in the 

baseline is consistent with the estimate.

• The documentation provides evidence that the cost estimate was reviewed and 

accepted by management.

Accurate • The cost estimate results are unbiased, not overly conservative or optimistic and 

based on an assessment of most likely costs.

• The estimate has been adjusted properly for inflation.

• The estimate contains few, if any, minor mistakes.

• The cost estimate is regularly updated to reflect significant changes in the program so 

that it is always reflecting current status. Variances between planned and actual costs 

are documented, explained, and reviewed.

• The estimate is based on a historical record of cost estimating and actual experiences 

from other comparable programs. 

Credible • The cost estimate includes a sensitivity analysis that identifies a range of possible 

costs based on varying major assumptions, parameters, and data inputs.

• A risk and uncertainty analysis was conducted that quantified the imperfectly 

understood risks and identified the effects of changing key cost driver assumptions 

and factors.

• Major cost elements were cross-checked to see whether results were similar.

• An independent cost estimate was conducted by a group outside the acquiring 

organization to determine whether other estimating methods produce similar results.

Source: GAO.
9 GAO-13-826R VXX Prototyping Waiver

Enclosure-2

Comments from the Department of Defense

(121169)