Page 1 GAO-12-983R EPS Prototyping Waiver

United States Government Accountability Office

Washington, DC 20548

August 23, 2012

Congressional Committees 

Subject: Department of Defense’s Waiver of Competitive Prototyping Requirement for 

Enhanced Polar System Program

The Weapon Systems Acquisition Reform Act of 2009, as amended (WSARA), requires that the 

acquisition strategy for each major defense acquisition program provides for competitive 

prototypes before Milestone B approval—which authorizes entry into system development—

unless the Milestone Decision Authority waives the requirement.1 Competitive prototyping, 

which involves commercial, government, or academic sources producing early prototypes of 

weapon systems or critical subsystems, can help Department of Defense (DOD) programs 

reduce technical risk, refine requirements, validate designs and cost estimates, and evaluate 

manufacturing processes prior to making major commitments of resources. WSARA states that 

the Milestone Decision Authority may waive the competitive prototyping requirement only on the 

basis that (1) the cost of producing competitive prototypes exceeds the expected life-cycle 

benefits (in constant dollars) of producing such prototypes, including the benefits of improved 

performance and increased technological and design maturity that may be achieved through 

competitive prototyping; or (2) but for such a waiver, DOD would be unable to meet critical 

national security objectives. WSARA also directed us to review and assess DOD’s rationale for 

competitive prototyping waivers.2

On June 18, 2012, DOD notified us that it had waived the competitive prototyping requirement

for the Control and Planning Segment (CAPS), a subsystem of the Air Force’s Enhanced Polar 

 

1

Pub. L. No. 111-23, § 203(a), as amended by the Ike Skelton National Defense Authorization Act for Fiscal Year 

2011, Pub. L. No. 111-383, § 813. Specifically, WSARA required DOD to modify its guidance relating to the operation 

of its acquisition system to incorporate these competitive prototyping provisions. DOD did so through Directive-Type 

Memorandum (DTM) 09-027, “Implementation of Weapon System Acquisition Reform Act of 2009 (Dec. 4, 2009, 

incorporating Change 3, Dec. 9, 2011). Major defense acquisition programs are those estimated by DOD to require 

an eventual total expenditure for research, development, test, and evaluation of more than $365 million, or for 

procurement, including all planned increments or spirals, of more than $2.19 billion in fiscal year 2000 constant 

dollars. The Milestone Decision Authority for major defense acquisition programs is the Under Secretary of Defense 

for Acquisition, Technology and Logistics, the head of a DOD component, or if delegated the component acquisition 

executive.

2

Pub. L. No. 111-23 § 203(b)(2). Specifically, WSARA provides that whenever a Milestone Decision Authority 

authorizes a waiver of the competitive prototyping requirement on the basis of what WSARA describes as “excessive 

cost,” the Milestone Decision Authority is required to submit notification of the waiver, together with the rationale, to 

the Comptroller General of the United States at the same time it is submitted to the congressional defense 

committees. WSARA further provides that we are to review the rationale for the waiver and submit a written 

assessment of that rationale to the congressional defense committees.
Page 2 GAO-12-983R EPS Prototyping Waiver

System (EPS).3 EPS, which consists of two payloads hosted on classified satellites, a gateway 

to connect user terminals to other communication systems, and a control and planning segment 

to control the payloads and manage communications, will provide extremely high frequency, 

jam-resistant, and secure satellite communications to strategic and tactical forces in the polar

region. According to DOD, this is the first waiver of WSARA’s competitive prototyping 

requirement since it was enacted in 2009. 

In this report, we assess (1) DOD’s rationale for waiving the competitive prototyping requirement 

for CAPS and the analysis used to support it and (2) the acquisition strategy for CAPS in the 

context of the waiver. To conduct our assessment, we compared the rationale in the waiver to 

the WSARA requirement to determine the extent to which the waiver is consistent with the 

statute. In addition, we reviewed the Air Force’s business case analysis, which provides the 

data and assumptions on which the waiver is based, the acquisition strategy, and other relevant

documentation. We also submitted written questions to DOD, Air Force, and EPS program 

officials to clarify information in this documentation, as necessary.

We conducted this performance audit from June 2012 to August 2012 in accordance with 

generally accepted government auditing standards. These standards require that we plan and 

perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on our audit objectives. 

Results in Brief

DOD’s rationale for waiving WSARA’s competitive prototyping requirement for CAPS covered 

both bases provided in the statute; however, DOD did not provide complete information about 

the potential benefits of competitive prototyping or support for its conclusion that prototyping 

would result in schedule delays. In the waiver, DOD found reasonable the Air Force’s 

conclusion that the additional $49 million cost of producing competing prototypes exceeded the 

negligible expected life-cycle benefits because minimal opportunities exist to increase CAPS 

technological and design maturity through competitive prototyping. The Air Force’s conclusions 

about CAPS technical and design risk are supported by its market research, but its cost-benefit 

analysis was incomplete because neither the waiver nor the business case analysis supporting 

it provided an estimated dollar value for the expected benefits. Further, the Air Force’s cost 

estimate of competitively prototyping CAPS was based on a program office estimate, which has 

not been independently reviewed by DOD’s Office of Cost Assessment and Performance 

Evaluation. DOD also did not provide support in the waiver for its conclusion that implementing 

competitive prototyping will delay EPS’s initial operational capability and result in DOD being 

unable to meet critical national security objectives. Finally, the CAPS acquisition strategy 

anticipates the use of a cost-reimbursement contract for designing and building the system; 

however it may be difficult for DOD to meet the requirements for justifying the use of a cost-type

contract for development, given the low design and development risk described in the waiver.

 

3

DOD reported in the waiver that the program has produced a single prototype. Pub. L. No. 111-23 § 203(a)(3)(A) 

provides that whenever a Milestone Decision Authority authorizes a competitive prototyping waiver, the program is 

still required to produce a prototype prior to Milestone B approval if the expected life-cycle benefits (in constant 

dollars) of producing such prototype exceed its cost and its production is consistent with achieving critical national 

security objectives.
Page 3 GAO-12-983R EPS Prototyping Waiver

Waiver Rationale Is Consistent with WSARA, but Gaps Exist in the Analysis

DOD’s rationale for waiving WSARA’s competitive prototyping requirement for CAPS addressed

the two bases provided for a waiver in the statute, which focus on costs and benefits and critical 

national security objectives. In the waiver, DOD accepted the Air Force’s conclusion that 

competitively prototyping CAPS would increase development costs by approximately $49 million

(base year 2012 constant dollars).4 Based on market research and other factors, such as 

previous efforts to reduce CAPS requirements and risk reduction work that demonstrated these 

requirements can be met with a low risk design and primarily commercial off-the-shelf hardware 

and software, the Air Force concluded that the life-cycle benefits of competitive prototyping were 

negligible and the level of cost savings needed to recoup the additional investment was

unrealistic.5

The Air Force’s conclusions about CAPS technical and design risk are supported by its market 

research, but its cost-benefit analysis was incomplete. Neither the DOD waiver nor the Air Force 

business case analysis included an estimate of the life-cycle benefits that may be achieved 

through competitive prototyping. In the absence of a specific dollar value or range of values for 

the estimated life-cycle benefits of CAPS competitive prototyping, the Air Force cannot 

conclusively determine whether the costs outweigh the benefits. According to the EPS program 

office, it did not perform a more detailed life-cycle cost-benefit analysis because the additional 

cost and burden of producing such an analysis was not warranted based on its business case 

analysis. Further, the Air Force’s estimated $49 million cost for competitively prototyping was 

based on a program office estimate and has not been independently estimated or reviewed by 

DOD’s cost estimating organization, the Office of Cost Assessment and Performance 

Evaluation. According to our best practices criteria, an independent cost estimate is considered 

one of the best and most reliable resource estimate validation methods because it provides an 

independent view of expected program costs that tests the program office estimate for 

reasonableness.6

DOD’s conclusion that it cannot meet critical national security objectives without the CAPS 

competitive prototyping waiver was not well supported in the waiver. The waiver states that 

implementing competitive prototypes for CAPS will further delay the EPS’s initial operational 

capability, but it does not explain why or how it adds to the schedule risks the program already 

faces. We noted during our review that the Air Force’s acquisition strategy concludes that 

competitive prototyping could delay initial operational capability by 6 months to 2 years because

 

4

According to the EPS program office, $49 million includes not only the cost of producing an additional prototype, but 

also the cost of funding a second contractor through preliminary design review.

5

During our review, we found errors in the program costs and percentages cited in the waiver. The waiver states that 

the Air Force estimate for remaining CAPS development following prototyping is $177 million and for CAPS 

operations and sustainment is $134 million (base year 2012 constant dollars). By comparing these costs to the cost 

of competitive prototyping, DOD concluded that a cost savings of more than 16 percent would have to be achieved to 

make competitive prototyping viable. However, the waiver incorrectly stated that these cost estimates were in base 

year 2012 constant dollars. We found and the Air Force acknowledged that the dollar amounts were in then-year 

dollars. During our review, we requested and received these dollars amounts in 2012 base year constant dollars from 

the EPS program office. It also provided us an updated estimate of operations and sustainment costs. According to 

this data, the estimated cost of CAPS development work following prototyping is $158 million and CAPS operations 

and sustainment is $71 million (base year 2012 constant dollars).

6

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program 

Costs, GAO-09-3SP (Washington, D.C.: March 2009).
Page 4 GAO-12-983R EPS Prototyping Waiver

the program would have to not only award contracts for competitive prototyping, but also go 

through another source selection to award a CAPS design and development contract. On the 

other hand, according to the EPS program office, alternatives exist if CAPS is not ready by the 

time the payloads, which have almost completed production, are launched. According to the 

program office, ground test assets have been procured to enable payload calibration and checkout prior to CAPS operational readiness, though these assets could not sustain CAPS 

operations. In addition, there are currently two Interim Polar System payloads in operation that 

the program office estimates will continue to meet operational requirements while the EPS is 

deployed and readied for operations.

Contract Type in CAPS Acquisition Strategy May Be Difficult to Justify 

The CAPS acquisition strategy anticipates the use of a cost-reimbursement contract for 

designing and building the system; however, it may be difficult for DOD to meet the 

requirements for justifying the use of a cost-type contract for development, given the description 

of risk in the waiver. The waiver concluded that, based on market research, there is no 

significant technology development anticipated in the contractors’ proposals for CAPS and 

design risks are expected to be low. However, DOD acquisition regulations state that if a costtype contract is selected for a development program at Milestone B, the contract file shall 

include the Milestone Decision Authority’s written determination that the program is so complex 

and technically challenging that it would not be practicable to reduce program risk to a level that 

would permit the use of a fixed-price type contract.7 As shown in figure 1, the EPS program’s 

Milestone B is planned for the first quarter of fiscal year 2014.

Figure 1: EPS and CAPS Program Schedule

Notes: CDR critical design review

FY fiscal year 

PDR preliminary design review

Q quarter 

According to the acquisition strategy, a cost-reimbursement contract was chosen for the basic 

CAPS contract because of the nature of the design work, which includes software design and 

development. This contract type was also chosen for the option to develop and build the system 

because of the difficulty in pricing the development of a system that has not yet completed 

 

7

Subpart 16.1 of the FAR and DFARS §§ 216.104-70 and § 234.004(2)(iii) outline a variety of factors to consider in 

selecting contract types.
Page 5 GAO-12-983R EPS Prototyping Waiver

preliminary design review. According to the acquisition strategy, using a cost-plus-incentive fee 

contract will reduce the level of risk for the competing contractors and allow them to deliver 

more reasonably priced proposals up front; however, it will result in the government assuming a 

greater portion of the cost and performance risk. Decisions on the specific contract type remain 

and in making those decisions, DOD will need to appropriately address the expected level of 

risk to both the government and contractors.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. In its written comments, reproduced in 

the enclosure, DOD agreed that it could have provided additional documentation in support of 

the waiver and stated that it appreciated our clarification of the types of information and data 

that should be included in the future waivers for best practices purposes. We believe that if DOD 

takes note of our findings on the CAPS waiver, in particular the lack of a specific estimate of the 

expected life-cycle cost benefits of competitive prototyping, it would improve future waiver 

requests. DOD also noted that WSARA does not specifically ask us to review waivers based on 

national security objectives and considers our comments on its national security rationale 

outside the scope of the statute. We recognize that WSARA specifically provides for our review 

of competitive prototyping waivers after receipt of a notification of a waiver on the basis of 

excessive cost, i.e., the cost of producing competitive prototypes exceeds the expected lifecycle benefits of producing such prototypes. However, in this instance, because DOD’s rationale 

for the waiver included both excessive cost and national security considerations, our 

assessment included both bases. Our finding that DOD’s national security rationale was not well 

supported still stands; and we believe that providing adequate support for the national security 

rationale in competitive prototyping waivers is important for congressional oversight. Finally, 

DOD noted that we questioned the justification for using a cost-reimbursement contract for 

CAPS. For clarification, we did not question DOD’s decision, which is ultimately one for the 

agency, on the appropriate contract type. Instead, we highlighted that the low design risks 

described in the waiver seem inconsistent with the requirements in DOD’s acquisition 

regulations for justifying the use of a cost-reimbursement contract for development.

- - - - - - 

We are sending copies of this report to interested congressional committees, the Secretary of 

Defense, and Secretary of the Air Force. In addition, the report will be available at no charge on 

the GAO website at http://www.gao.gov. 
Page 6 GAO-12-983R EPS Prototyping Waiver

If you or your staff have any questions, please contact me at (202) 512-4841 or 

sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs 

may be found on the last page of this report. GAO staff who made key contributions to this 

report were Ron Schwenn, Assistant Director; Jacob Leon Beier; Marie Ahearn; Kenneth E. 

Patton; and Carol Petersen.

Michael J. Sullivan

Director, Acquisition and Sourcing Management

Enclosure
Page 7 GAO-12-983R EPS Prototyping Waiver

List of Committees

The Honorable Carl Levin

Chairman

The Honorable John McCain

Ranking Member

Committee on Armed Services

United States Senate

The Honorable Daniel K. Inouye

Chairman

The Honorable Thad Cochran

Ranking Member

Subcommittee on Defense

Committee on Appropriations

United States Senate

The Honorable Howard P. “Buck” McKeon

Chairman

The Honorable Adam Smith

Ranking Member

Committee on Armed Services

House of Representatives

The Honorable C.W. Bill Young

Chairman

The Honorable Norman D. Dicks

Ranking Member

Subcommittee on Defense

Committee on Appropriations

House of Representatives
Page 8 GAO-12-983R EPS Prototyping Waiver

Enclosure

Comments from the Department of Defense
Page 9 GAO-12-983R EPS Prototyping Waiver

(121085)
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.