INFORMATION 

TECHNOLOGY

Agencies Need to 

Establish and 

Implement 

Incremental 

Development Policies

Report to the Committee on Homeland 

Security and Governmental Affairs, 

U.S. Senate

May 2014

GAO-14-361

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-14-361, a report to the

Committee on Homeland Security

and Governmental Affairs, U.S. Senate

May 2014

INFORMATION TECHNOLOGY

Agencies Need to Establish and Implement 

Incremental Development Policies

Why GAO Did This Study

Federal agencies plan to spend at 

least $82 billion on IT in fiscal year 

2014. However, prior IT expenditures 

have often produced disappointing 

results. Thus, OMB has called for 

agencies to deliver investments in 

smaller parts or increments. In 2010, it 

called for IT investments to deliver 

capabilities every 12 months and now 

requires investments to deliver 

capabilities every 6 months. GAO was 

asked to review agencies’ incremental 

development approaches. Among 

other things, this report (1) assesses

whether selected agencies have 

established policies for incremental IT 

development; and (2) determines

whether selected agencies are using 

incremental development approaches 

to manage their IT investments. To do 

so, GAO selected five agencies—

Defense, HHS, DHS, Transportation, 

and VA—and 89 total investments at 

these agencies. GAO then reviewed

the agencies’ incremental development 

policies and plans.

What GAO Recommends

Among other things, GAO 

recommends that OMB develop and 

issue realistic and clear guidance on 

incremental development and that the 

selected agencies update and 

implement their incremental 

development policies to reflect OMB’s 

guidance. OMB partially disagreed, 

believing its guidance is realistic. Four 

agencies generally agreed with the 

report or had no comments, and one 

agency did not agree that its 

recommendations should be

dependent on OMB first taking action. 

GAO continues to believe that its 

recommendations are valid, as 

discussed in this report.

What GAO Found

All five agencies in GAO’s review—the Departments of Defense (Defense), 

Health and Human Services (HHS), Homeland Security (DHS), Transportation 

(Transportation), and Veterans Affairs (VA)—have established policies that 

address incremental development; however, the policies usually did not fully 

address three key components for implementing the Office of Management and 

Budget’s (OMB) guidance (see table). Specifically, only VA fully addressed the 

three components. Among other things, agencies cited the following reasons that 

contributed to these weaknesses: (1) the guidance was not feasible because not 

all types of investments should deliver functionality in 6 months, and (2) the 

guidance did not identify what agencies’ policies are to include or time frames for 

completion. GAO agrees these concerns have merit. Until OMB issues realistic 

and clear guidance and agencies address the weaknesses in their incremental 

development policies, it will be difficult to deliver project capability more rapidly.

Assessment of Selected Agencies’ Incremental Development Policies

Component Defense HHS DHS Transportation VA

Require delivery of 

functionality every 6 

months

○ ○ ○ ○ ●

Define functionality ◐ ○ ○ ○ ●

Define a process for 

enforcing compliance ◐ ◐ ○ ○ ●

Key ●=Fully met ◐=Partially met ○=Not met

Source: GAO analysis of agency documentation.

The weaknesses in agency policies have enabled inconsistent implementation of 

incremental development approaches: almost three-quarters of the selected 

investments did not plan to deliver functionality every 6 months, and less than

half planned to deliver functionality in 12-month cycles (see table). Without 

consistent use of incremental development approaches, information technology 

(IT) expenditures are more likely to continue producing disappointing results.

Number of Selected Investments Planning to Incrementally Deliver Functionality

Agency

Total number 

of selected 

investments

Investments planning 

to deliver functionality 

every 6 months 

Investments planning

to deliver functionality 

every 12 months

Defense 37 1 11

HHS 14 9 11

DHS 12 2 6

Transportation 20 5 7

VA 6 6 6

Totals 89 23 41

Source: GAO analysis of agency data.

View GAO-14-361. For more information, 

contact David A. Powner at (202) 512-9286 or 

pownerd@gao.gov.
Page i GAO-14-361 Incremental Development 

Letter 1

Background 3

Majority of Selected Agencies’ Incremental Development Policies 

Have Significant Weaknesses 12

Most Selected Investments Do Not Plan to Deliver Functionality 

Every 6 or 12 Months 19

Multiple Factors Were Commonly Identified as Enabling and 

Inhibiting Incremental Development 26

Conclusions 33

Recommendations for Executive Action 34

Agency Comments and Our Evaluation 35

Appendix I Objectives, Scope, and Methodology 39

Appendix II Selected Investments’ Plans for Delivering Functionality 45

Appendix III Comments from the Department of Defense 49

Appendix IV Comments from the Department of Homeland Security 51

Appendix V Comments from the Department of Veterans Affairs 53

Appendix VI GAO Contact and Staff Acknowledgments 55

Tables

Table 1: Assessment of Selected Agencies’ Incremental 

Development Policies 14

Table 2: Number and Percentage of Selected Investments That 

Planned to Deliver Functionality Every 6 Months, by 

Agency, for Fiscal Years 2013 and 2014 19

Contents
Page ii GAO-14-361 Incremental Development 

Table 3: Number and Percentage of Remaining Selected 

Investments That Planned to Deliver Functionality Every 6 

Months, by Agency, for Fiscal Years 2013 and 2014 23

Table 4: Number and Percentage of Selected Investments That 

Planned to Deliver Functionality Every 12 Months, by 

Agency, for Fiscal Years 2013 and 2014 24

Table 5: Commonly Identified Factors Enabling Incremental 

Development during a 6-month Period 27

Table 6: Commonly Identified Factors Inhibiting Incremental 

Development during a 6-month Period 30

Table 7: Selected Investments and Associated Plans for 

Delivering Functionality Every 6 and 12 Months, for Fiscal 

Years 2013 and 2014 45

Abbreviations

CIO chief information officer

CMMI-ACQ Capability Maturity Model® Integration for 

Acquisition 

Defense Department of Defense

DHS Department of Homeland Security 

FAA Federal Aviation Administration

HHS Department of Health and Human Services

IT information technology

OMB Office of Management and Budget

SEI Software Engineering Institute

Transportation Department of Transportation

VA Department of Veterans Affairs

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-14-361 Incremental Development 

441 G St. N.W.

Washington, DC 20548

May 1, 2014

The Honorable Thomas R. Carper

Chairman

The Honorable Tom Coburn, M.D.

Ranking Member

Committee on Homeland Security and Governmental Affairs

United States Senate

Information systems are critical to the health, economy, and security of 

the nation. To support agency missions, the federal government plans to 

spend at least $82 billion on information technology (IT) investments in 

fiscal year 2014. However, as we have previously reported, prior IT 

expenditures too often have produced failed projects—that is, projects 

with multimillion dollar cost overruns and schedule delays measured in 

years, with questionable mission-related achievements.1

To help resolve these issues, Congress and the Office of Management 

and Budget (OMB) have called for agencies to deliver investments in 

smaller parts, or increments, in order to reduce investment risk, deliver 

capabilities more quickly, and facilitate the adoption of emerging 

technologies.

Many of these 

investments have been broadly scoped projects that aim to deliver their 

capabilities several years after initiation.

2 In 2010, OMB placed a renewed emphasis on incremental 

IT development by calling for major IT investments to deliver functionality 

at least every 12 months. Subsequently, OMB has made this guidance 

more stringent, and annual budget guidance now states that each project 

associated with major IT investments is to deliver functionality every 6 

months.3

 1

GAO, OMB and Agencies Need to More Effectively Implement Major Initiatives to Save 

Billions of Dollars, GAO-13-796T (Washington, D.C.: July 25, 2013).

2

41 U.S.C. § 2308; OMB, Management of Federal Information Resources, Circular No. 

A-130 Revised.

3

OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government 

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012); Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management of 

Information Technology Capital Assets (2011).
Page 2 GAO-14-361 Incremental Development 

This report responds to your request that we review agencies’ 

incremental IT development. Specifically, we (1) assessed whether 

selected agencies have established policies for incremental IT 

development, (2) determined whether selected agencies are using 

incremental development approaches to manage their IT investments, 

and (3) identified the key factors that enabled and inhibited the selected 

agencies’ abilities to effectively use incremental development approaches 

to manage their IT investments.

In conducting our review, we selected five agencies and 89 investments 

to review. We selected the five agencies with the largest IT budgets for 

development, modernization, and enhancement on major IT investments4

To address our first objective, we reviewed OMB guidance related to the 

use of incremental development and identified three key components of 

incremental development that agencies should include in their policies. 

Specifically, in order to identify these components, we reviewed OMB’s 

Circular A-130

in fiscal years 2013 and 2014: the Departments of Defense (Defense), 

Health and Human Services (HHS), Homeland Security (DHS), 

Transportation (Transportation), and Veterans Affairs (VA). We then 

selected the major IT investments at these agencies that planned to 

spend more than 50 percent of the investments’ fiscal year 2013 and 

2014 budgets on development, modernization, and enhancement. These 

89 investments are identified in appendix II.

5

—which requires agencies to develop incremental 

development policies, but does not provide guidance on what the policies 

are to include—as well as more recent OMB guidance6

 4

“Major IT investment” means a system or an acquisition requiring special management 

attention because it has significant importance to the mission or function of the 

government; significant program or policy implications; high executive visibility; high 

development, operating, or maintenance costs; an unusual funding mechanism; or is 

defined as major by the agency’s capital planning and investment control process.

on incremental 

development and leading industry guidance on institutionalizing 

5

OMB, Management of Federal Information Resources, Circular No. A-130 Revised.

6

OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Executive Office of the President of the United States, OMB, Fiscal Year 2014 

Analytical Perspectives: Budget of the U.S. Government, 354; OMB, Contracting 

Guidance to Support Modular Development (June 14, 2012); Guidance on Exhibits 53 and 

300—Information Technology and E-Government (2012); and Guidance on Exhibit 300—

Planning, Budgeting, Acquisition, and Management of Information Technology Capital 

Assets (2011).
Page 3 GAO-14-361 Incremental Development 

processes throughout an organization.7

To address our second objective, we analyzed information obtained from 

data collection instruments describing how often the selected investments 

planned to deliver functionality. Specifically, we administered a data 

collection instrument to officials responsible for each of the selected

investments that asked how often they planned to deliver functionality 

during fiscal years 2013 and 2014. Using these data, we determined the 

extent to which the selected investments and their projects are meeting 

OMB’s guidance on incremental development.

We then compared the selected 

agencies’ incremental development policies to the components we 

identified.

To address our third objective, officials from the five selected agencies 

and investments identified the key factors that have both enabled and 

inhibited their efforts to deliver functionality for their major IT investments, 

consistent with OMB guidance. We then determined the key factors that 

were commonly identified by the selected agencies and investments.

We conducted this performance audit from May 2013 to May 2014, in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives. Details of our objectives, 

scope, and methodology are contained in appendix I.

If done correctly, investments in IT have the potential to make 

organizations more efficient in fulfilling their missions. For example, we 

recently reported that Defense officials stated that an IT system 

supporting military logistics has improved the organization’s performance 

by providing real-time information about road conditions, construction, 

incidents, and weather to facilitate rapid deployment of military assets.8

 7

Software Engineering Institute (SEI), Capability Maturity Model® Integration for 

Acquisition (CMMI-ACQ), Version 1.3 (November 2010).

8

GAO, Information Technology: Additional Executive Review Sessions Needed to Address 

Troubled Projects, GAO-13-524 (Washington, D.C.: June 13, 2013).

Background
Page 4 GAO-14-361 Incremental Development 

However, as we have previously reported, investments in federal IT too 

frequently result in failed projects that incur cost overruns and schedule 

slippages while contributing little to mission-related outcomes. For 

example:

• In January 2011, the Secretary of Homeland Security ended the 

Secure Border Initiative Network program after obligating more than 

$1 billion for the program because it did not meet cost-effectiveness 

and viability standards. Since 2007, we have identified a range of 

issues and made several recommendations to improve this program.9

For example, in May 2010, we reported that the final acceptance of 

the first two deployments had slipped from November 2009 and 

March 2010 to September 2010 and November 2010, respectively, 

and that the cost-effectiveness of the system had not been justified.10

We concluded that DHS had not demonstrated that the considerable 

time and money being invested to acquire and deploy the program 

were a wise and prudent use of limited resources. As a result, we 

recommended that the department (1) limit near-term investment in 

the program, (2) economically justify any longer-term investment in it, 

and (3) improve key program management disciplines. This work 

contributed to the department’s decision to cancel the program.

• In February 2011, the Office of Personnel Management canceled its 

Retirement Systems Modernization program after several years of 

trying to improve the implementation of this investment.11

 9

See, for example, GAO, Secure Border Initiative: DHS Needs to Strengthen Management 

and Oversight of Its Prime Contractor, 

According to 

the Office of Personnel Management, it spent approximately $231 

million on this investment. We issued a series of reports on the 

GAO-11-6 (Washington, D.C.: Oct. 18, 2010); 

Secure Border Initiative: DHS Needs to Reconsider Its Proposed Investment in Key 

Technology Program, GAO-10-340 (Washington, D.C.: May 5, 2010); Secure Border 

Initiative: DHS Needs to Address Testing and Performance Limitations That Place Key 

Technology Program at Risk, GAO-10-158 (Washington, D.C.: Jan. 29, 2010); Secure 

Border Initiative: DHS Needs to Address Significant Risks in Delivering Key Technology 

Investment, GAO-08-1086 (Washington, D.C.: Sept. 22, 2008); and Secure Border 

Initiative: SBInet Expenditure Plan Needs to Better Support Oversight and Accountability, 

GAO-07-309 (Washington, D.C.: Feb. 15, 2007).

10GAO-10-340.

11GAO, OPM Retirement Modernization: Longstanding Information Technology 

Management Weaknesses Need to Be Addressed, GAO-12-226T (Washington, D.C.: 

Nov. 15, 2011).
Page 5 GAO-14-361 Incremental Development 

agency’s efforts to modernize its retirement system and found that the 

Office of Personnel Management was hindered by weaknesses in 

several important management disciplines that are essential to 

successful IT modernization efforts.12 Accordingly, we made 

recommendations in areas such as project management, 

organizational change management, testing, and cost estimating. In 

May 2008, an Office of Personnel Management official cited the 

issues that we identified as justification for issuing a stop work order 

to the system contractor, and the agency subsequently terminated the 

contract.

• In December 2012, Defense canceled the Air Force’s Expeditionary 

Combat Support System after having spent more than a billion dollars 

and missing multiple milestones, including failure to achieve 

deployment within 5 years of obligating funds. We issued several 

reports on this system and found that, among other things, the 

program was not fully following best practices for developing reliable 

schedules and cost estimates.13

Agencies have reported that poor-performing projects have often used a 

“big bang” approach—that is, projects that are broadly scoped and aim to 

deliver functionality several years after initiation. For example, in 2009 the 

Defense Science Board reported that Defense’s acquisition process for IT 

systems—which was rooted in the “waterfall” development model14

 12GAO, Office of Personnel Management: Retirement Modernization Planning and 

Management Shortcomings Need to Be Addressed, 

—was 

GAO-09-529 (Washington, D.C.: Apr. 

21, 2009); Office of Personnel Management: Improvements Needed to Ensure Successful 

Retirement Systems Modernization, GAO-08-345 (Washington, D.C.: Jan. 31, 2008); 

Comments on the Office of Personnel Management’s February 20, 2008 Report to 

Congress Regarding the Retirement Systems Modernization, GAO-08-576R (Washington, 

D.C.: Mar. 28, 2008); and Office of Personnel Management: Retirement Systems 

Modernization Program Faces Numerous Challenges, GAO-05-237 (Washington, D.C.: 

Feb. 28, 2005).

13GAO, DOD Business Transformation: Improved Management Oversight of Business 

System Modernization Efforts Needed, GAO-11-53 (Washington, D.C.: Oct. 7, 2010); and 

DOD Financial Management: Implementation Weaknesses in Army and Air Force 

Business Systems Could Jeopardize DOD’s Auditability Goals, GAO-12-134 (Washington, 

D.C.: Feb. 28, 2012).

14The waterfall model begins with requirements development and continues sequentially 

through other phases—design, build, and testing—using the output of one phase as the 

input to the next to develop a finished product at the end. There is only one product 

release at the end of testing.
Page 6 GAO-14-361 Incremental Development 

too long, ineffective, and did not accommodate the rapid evolution of IT.15

One approach to reducing the risks from broadly scoped, multiyear 

projects is to divide investments into smaller parts—a technique long 

advocated by Congress and OMB.

The board reported that the average time to deliver an initial program 

capability for a major IT system acquisition at Defense was over 7 years. 

Also in 2009, VA’s former chief information officer (CIO) reported that 

many of its projects exceeded cost estimates by more than 50 percent 

and missed scheduled completion dates by more than a year. That official 

concluded that VA needed to make substantial changes to its acquisition 

process in order to eliminate project failures associated with the “big 

bang” approach.

16

• deliver capabilities to their users more rapidly, giving them more 

flexibility to respond to changing agency priorities;

By following this approach, agencies 

can potentially

• increase the likelihood that each project will achieve its cost, 

schedule, and performance goals;

• obtain additional feedback from users, increasing the probability that 

each successive increment and project will meet user needs;

• more easily incorporate emerging technologies; and

• terminate poorly performing investments with fewer sunk costs.

OMB has placed a renewed emphasis on such development. Since 2000, 

OMB Circular A-130 has required agencies to incorporate incremental 

development approaches into their policies and ensure that investments 

 15Defense, Defense Science Board, Report of the Defense Science Board Task Force on 

Department of Defense Policies and Procedures for the Acquisition of Information 

Technology (Washington, D.C.: March 2009).

16See Clinger-Cohen Act of 1996, Pub. L. No. 104-106 § 5202, 110 Stat. 186, 690 (1996), 

codified at 41 U.S.C. § 2308; see also 48 C.F.R. § 39.103 (Federal Acquisition 

Regulation); OMB, Management of Federal Information Resources, Circular No. A-130 

Revised.
Page 7 GAO-14-361 Incremental Development 

implement them.17 More recently, in its 2010 IT Reform Plan, OMB called 

for IT programs to deliver functionality at least every 12 months and 

complete initial deployment to end users no later than 18 months after the 

start of the program.18 In 2011, as part of its budget guidance, OMB first 

recommended that projects associated with major IT investments deliver 

functionality every 6 months.19 OMB’s latest guidance now makes this 

mandatory; specifically, in 2012, OMB began requiring that functionality 

be delivered at least every 6 months.20

Over the last three decades, Congress has enacted several laws to assist 

agencies and the federal government in managing IT investments. For 

example, the Paperwork Reduction Act of 1995 requires that OMB 

develop and oversee policies, principles, standards, and guidelines for 

federal agency IT functions, including periodic evaluations of major 

information systems.21 In addition, to assist agencies in managing their 

investments, Congress enacted the Clinger-Cohen Act of 1996.22 Among 

other things, the act requires agency heads to appoint CIOs and specifies 

many of their responsibilities.23

 17OMB, Management of Federal Information Resources, Circular No. A-130 Revised. 

OMB’s 2012 and 2013 guidance reaffirmed and strengthened these requirements. 

Executive Office of the President of the United States, OMB, Fiscal Year 2014 Analytical 

Perspectives: Budget of the U.S. Government, 354; and OMB, Contracting Guidance to 

Support Modular Development (June 14, 2012).

With regard to IT management, CIOs are 

responsible for implementing and enforcing applicable governmentwide 

and agency IT management principles, standards, and guidelines; 

assuming responsibility and accountability for IT investments; and 

monitoring the performance of IT programs and advising the agency head 

18OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010).

19OMB, Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management of 

Information Technology Capital Assets (2011).

20OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012).

2144 U.S.C. § 3501, et. seq.

2240 U.S.C. § 11101, et. seq.

2344 U.S.C. § 3506.

Roles and Responsibilities 

for Overseeing Federal IT 

Investments
Page 8 GAO-14-361 Incremental Development 

whether to continue, modify, or terminate such programs.24 Additionally, 

with regard to incremental development, Clinger-Cohen calls for 

provisions in the Federal Acquisition Regulation that encourage agencies 

to structure their IT contracts such that the capabilities are delivered in 

smaller increments. The Federal Acquisition Regulation provisions are 

also to provide, to the maximum extent practicable, that the increment 

should be delivered within 18 months of the contract solicitation.25

As set out in these laws, OMB is to play a key role in helping federal 

agencies manage their investments by working with them to better plan, 

justify, and determine how much they need to spend on projects and how 

to manage approved projects. Within OMB, the Office of E-Government 

and Information Technology, headed by the Federal CIO,26

In carrying out its responsibilities, OMB uses several data collection 

mechanisms to oversee federal IT spending during the annual budget 

formulation process. Specifically, OMB requires federal departments and 

agencies to provide information related to their IT investments (called 

exhibit 53s) and capital asset plans and business cases (called exhibit 

300s).

directs the 

policy and strategic planning of federal IT investments and is responsible 

for oversight of federal technology spending.

• Exhibit 53. The purpose of the exhibit 53 is to identify all IT 

investments—both major and nonmajor—and their associated costs 

within a federal organization. Information included in agency exhibit 

53s is designed, in part, to help OMB better understand agencies’ 

spending on IT investments.

• Exhibit 300. The purpose of the exhibit 300 is to provide a business 

case for each major IT investment and to allow OMB to monitor IT 

investments once they are funded. An IT investment may include one 

 2440 U.S.C. § 11315.

2541 U.S.C. § 2308. The Federal Acquisition Regulation was amended in February 1998 

to reflect this provision. 48 C.F.R. § 39.103. Of note, the Federal Aviation Administration 

(FAA), part of Transportation, is exempt from the Federal Acquisition Regulation. 49 

U.S.C. § 40110 (d)(2)(G). Instead, FAA operates under an independent acquisition 

system, referred to as FAA’s Acquisition Management System.

26The Federal CIO is the presidential designation for the Administrator of the OMB Office 

of E-Government.
Page 9 GAO-14-361 Incremental Development 

or more projects that are to develop, modernize, enhance, or maintain 

a single IT asset or group of IT assets with related functionality. 

Agencies are required to provide information on each major 

investment’s projects, including cost, schedule, and performance 

information. For example, in order to measure compliance with its 

requirement that projects deliver functionality in 6-month cycles, OMB 

requires agencies to break their projects into activities, and describe 

when the activities are to deliver functionality.

OMB has implemented a series of initiatives to improve the oversight of 

underperforming investments and more effectively manage IT. These 

efforts include the following:

• IT Dashboard. In June 2009, to further improve the transparency into 

and oversight of agencies’ IT investments, OMB publicly deployed the 

IT Dashboard.27 As part of this effort, OMB issued guidance directing 

federal agencies to report, via the Dashboard, the performance of 

their IT investments. Currently, the Dashboard publicly displays

information on the cost, schedule, and performance of over 700 major 

federal IT investments at 26 federal agencies. Further, the public 

display of these data is intended to allow OMB, other oversight 

bodies, and the general public to hold the government agencies 

accountable for results and progress.

• TechStat reviews. In January 2010, the Federal CIO began leading 

TechStats sessions—face-to-face meetings to terminate or turn 

around IT investments that are failing or are not producing results. 

These meetings involve OMB and agency leadership and are 

intended to increase accountability and transparency and improve 

performance. For example, the Federal CIO testified in June 2013 that 

he holds TechStat meetings on large investments that are not being 

acquired incrementally.28

 27

More recently, OMB empowered agency 

CIOs to hold their own TechStat sessions within their respective 

agencies. In doing so, OMB has called for agencies to use their 

http://www.itdashboard.gov.

28Reducing Duplication and Improving Outcomes in Federal Information Technology, 

Before the S. Comm. on Homeland Security and Governmental Affairs, 113 Cong. 23 

(2013) (statement of Federal CIO Steven VanRoekel).

OMB’s Recent Major 

Initiatives for Overseeing 

IT Investments
Page 10 GAO-14-361 Incremental Development 

TechStat processes to identify investments that are not being 

acquired incrementally and undertake corrective actions.29

• IT Reform Plan. In December 2010, OMB released its 25-point plan to 

reform federal IT.30 This document established an ambitious plan for 

achieving operational efficiencies and effectively managing largescale IT programs. In particular, as part of its effort to effectively 

manage IT acquisitions, the plan calls for federal IT programs to 

deploy functionality in release cycles no longer than 12 months, and 

ideally, less than 6 months. The plan also identifies key actions that 

can help agencies implement this incremental development guidance, 

such as working with Congress to develop IT budget models that align 

with incremental development, and issuing contracting guidance and 

templates to support incremental development.

In April 2012, we reported on OMB’s efforts to implement the actions 

called for in its IT Reform Plan and found that it had partially completed 

work on two key action items relating to incremental development—

issuing contracting guidance and templates to support incremental 

development and working with Congress to create IT budget models that 

align with incremental development.31

 29Executive Office of the President of the United States, OMB, Fiscal Year 2014 Analytical 

Perspectives: Budget of the U.S. Government, 354.

With respect to the contracting 

guidance and templates, we found that, although OMB worked with the IT 

and acquisition community to develop guidance, it had not yet issued this 

guidance or the templates. Regarding the IT budget models, we found 

that, although OMB worked to promote ideas for IT budget flexibility (such 

as multiyear budgets or revolving funds) with congressional committees, 

there has not yet been any new legislation to create budget models, and 

OMB has not identified options to increase transparency for programs 

that would fall under these budgetary flexibilities. We recommended that 

the Director of OMB ensure that all action items called for in the IT 

Reform Plan are completed. OMB agreed with this recommendation. 

OMB has since issued contracting guidance for incremental 

30OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010).

31GAO, Information Technology Reform: Progress Made; More Needs to Be Done to 

Complete Actions and Measure Results, GAO-12-461 (Washington, D.C.: Apr. 26, 2012).

GAO Has Reported on 

Efforts to Improve IT 

Acquisitions
Page 11 GAO-14-361 Incremental Development 

development,32

Additionally, in 2011, we identified seven successful investment 

acquisitions and nine common factors critical to their success.

but, as of January 2014, a staff member from the OMB 

Office of E-Government and Information Technology stated that activities 

to address the development of new IT budget models are still ongoing.

33

Specifically, we reported that department officials identified seven 

successful investment acquisitions, in that they best achieved their 

respective cost, schedule, scope, and performance goals.34

 32OMB, Contracting Guidance to Support Modular Development (June 14, 2012).

Notably, all of 

these were smaller increments, phases, or releases of larger projects. For 

example, the Defense investment in our sample was the seventh 

increment of an ongoing investment; the Department of Energy system 

was the first of two phases; the DHS investment was rolled out to two 

locations prior to deployment to 37 additional locations; and the 

Transportation investment had been part of a prototype deployed to four 

airports. In addition, common factors critical to the success of three or 

more of the seven investments were: (1) program officials were actively 

engaged with stakeholders, (2) program staff had the necessary 

knowledge and skills, (3) senior department and agency executives 

supported the programs, (4) end users and stakeholders were involved in 

the development of requirements, (5) end users participated in testing of 

system functionality prior to formal end user acceptance testing, (6) 

government and contractor staff were stable and consistent, (7) program 

staff prioritized requirements, (8) program officials maintained regular 

communication with the prime contractor, and (9) programs received 

sufficient funding. These critical factors support OMB’s objective of 

improving the management of large-scale IT acquisitions across the 

federal government, and wide dissemination of these factors could 

complement OMB’s efforts.

33GAO, Information Technology: Critical Factors Underlying Successful Major 

Acquisitions, GAO-12-7 (Washington, D.C.: Oct. 21, 2011).

34The seven investments were (1) Department of Commerce’s Decennial Response 

Integration System, (2) Defense’s Global Combat Support System Joint (Increment 7), (3) 

Department of Energy’s Manufacturing Operations Management Project, (4) DHS’s 

Western Hemisphere Travel Initiative, (5) Transportation’s Integrated Terminal Weather 

System, (6) Internal Revenue Service’s Customer Account Data Engine 2, and (7) VA’s 

Occupational Health Record-keeping System.
Page 12 GAO-14-361 Incremental Development 

Further, in 2012, we identified 32 practices and approaches as effective 

for applying Agile software development35 methods to IT projects.36

Officials from five agencies who had used Agile methods on federal 

projects cited beneficial practices, such as obtaining stakeholder and 

customer feedback frequently, managing requirements, and ensuring staff 

had the proper knowledge and experience. We also identified 14 

challenges with adapting and applying Agile in the federal environment, 

including agencies having difficulty with committing staff to projects, 

procurement practices that did not support Agile projects, and compliance 

reviews that were difficult to execute within an iterative time frame. We 

noted that the effective practices and approaches identified in the report, 

as well as input from others with broad Agile experience, could help 

agencies in the initial stages of adopting Agile.

Since 2000, OMB Circular A-130 has required agencies to (1) develop 

policies that require their major investments to deliver functionality 

incrementally and (2) ensure that investments comply with their policies.37

In addition, as part of its recent budget guidance, OMB has defined how 

often investments must deliver functionality. Specifically, each project 

associated with major IT investments is to deliver functionality at least 

once every 6 months.38

 35Agile development calls for the delivery of software in small, short increments rather 

than in the typically long, sequential phases of a traditional waterfall approach. More a 

philosophy than a methodology, Agile emphasizes early and continuous software delivery, 

as well as using collaborative teams and measuring progress with working software. The 

Agile approach was first articulated in a 2001 document called the Agile Manifesto, which 

is still used today. The manifesto has four values: (1) individuals and interactions over 

processes and tools, (2) working software over comprehensive documentation, (3) 

customer collaboration over contract negotiation, and (4) responding to change over 

following a plan.

Further, through the President’s Budget, OMB 

has provided additional guidance on how incremental development is to 

36GAO, Software Development: Effective Practices and Federal Challenges in Applying 

Agile Methods, GAO-12-681 (Washington, D.C.: July 27, 2012).

37OMB, Management of Federal Information Resources, Circular No. A-130 Revised. In 

2012, OMB reaffirmed its call for agencies to develop policies on incremental 

development, but it did not specify what these policies are to include. OMB, Contracting 

Guidance to Support Modular Development (June 14, 2012).

38OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012); Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management of 

Information Technology Capital Assets (2011).

Majority of Selected 

Agencies’ Incremental 

Development Policies 

Have Significant 

Weaknesses
Page 13 GAO-14-361 Incremental Development 

be enforced by requiring agencies to use their TechStat processes to 

identify investments that are not being acquired incrementally and 

undertake corrective actions.39

Although OMB’s guidance requires agencies to develop incremental 

development policies, it does not specify what those policies are to 

include. Absent this detail and in reviewing the previously mentioned 

guidance and leading practices on institutionalizing processes throughout 

an organization,40

• require that all projects associated with major IT investments deliver 

functionality in cycles that are not more than 6-months long;

we identified three components that agencies should 

include in their policies in order to effectively carry out OMB’s incremental 

development guidance:

• define functionality—that is, what the projects are to deliver at the end 

of a 6-month cycle; and

• define a process for ensuring that major IT investments and their 

projects deliver functionality every 6 months. This should include 

identifying investments that are not being acquired incrementally 

through agency TechStat processes and undertaking corrective 

actions.

Although all five selected agencies developed policies that address 

incremental development, the majority of the agencies’ policies did not 

fully address all three components. Specifically, only VA fully addressed 

the three components; Defense partially addressed the majority of them; 

and HHS, DHS, and Transportation did not address the majority of 

components.

Table 1 provides a detailed assessment of each agency’s policies against 

the three key components of an incremental development policy. In 

addition, a discussion of each policy component follows the table.

 39Executive Office of the President of the United States, OMB, Fiscal Year 2014 Analytical 

Perspectives: Budget of the U.S. Government, 354.

40SEI, CMMI-ACQ, Version 1.3 (November 2010).
Page 14 GAO-14-361 Incremental Development 

Table 1: Assessment of Selected Agencies’ Incremental Development Policies

Agency

Component Defense HHS DHS Transportation VA

Require delivery of functionality 

every 6 months ○ ○ ○ ○ ●

Define functionality ◐ ○ ○ ○ ●

Define a process for enforcing 

compliance ◐ ◐ ○ ○ ●

Key:

●=Fully met—the agency provided evidence that addressed the component.

◐=Partially met—the agency provided evidence that addressed about half or a large portion of the 

component.

○=Not met—the agency did not provide evidence that addressed the component or provided 

evidence that minimally addressed the component.

Source: GAO analysis of agency documentation.

• Require delivery of functionality every 6 months. Only one of the five 

agencies—VA—fully addressed this policy component by clearly 

requiring that its projects be completed in increments that must not 

exceed 6 months.41 The other four agencies did not address this 

policy component. Three of these agencies—Defense, HHS, and 

DHS—all developed policies that promote the use of incremental 

development, but these policies do not require functionality to be 

delivered every 6 months. Specifically, with regard to Defense, 

although the department’s acquisition framework calls for investments 

to use incremental development,42 its policy on IT budget submissions 

encourages investments to deliver functionality every 12-18 months—

not every 6 months.43 According to officials of the Defense Office of 

the CIO, 12-18 month incremental development is better aligned with 

the acquisition framework that many of its IT acquisitions have used.44

 41VA, Project Management Accountability System, Directive 6071 (Feb. 20, 2013).

42Defense, Operation of the Defense Acquisition System, Interim Instruction No. 5000.02 

(Nov. 26, 2013).

43See, for example, Defense, Office of the Secretary of Defense, Guidance for Fiscal Year 

2013 IT Budget Submissions (Aug. 9, 2011).

44Using this same rationale, Defense has requested that OMB grant the department an 

exemption from OMB’s 6-month requirement for the past 3 years. According to OMB staff 

members from the Office of E-Government, OMB has not granted these requests.
Page 15 GAO-14-361 Incremental Development 

For HHS, although its policy requires incremental development, its 

policy recommends—but does not require—that all projects deliver 

functionality every 3-6 months. According to an HHS Office of the CIO 

official, HHS does not require its projects to deliver functionality every 

6 months because it wants to provide projects with flexibility. For 

DHS, in June 2012, the former DHS CIO issued a draft policy 

encouraging IT projects to move towards Agile development 

approaches. Additionally, with respect to financial systems 

modernization programs, DHS’s policy calls for providing financial 

capabilities to the customer in small increments of 6-12 months.45

According to DHS officials representing the Office of the CIO, the 

department is currently developing a departmentwide policy on 

incremental development; however, they said that the draft currently 

encourages investments to deliver the first release 18 months after 

program initiation and thereafter deploy functionality in cycles no 

longer than 12 months, but ideally less than 6 months. Lastly, 

Transportation also did not address the component because, although 

Transportation has a policy that calls for projects to deliver 

functionality every 6 months, officials from the Office of the CIO 

explained that this policy does not apply to the Federal Aviation 

Administration (FAA).46 These officials explained that how often FAA 

projects deliver functionality depends on their size, scope, risk, 

visibility, and interdependencies with other programs.

• Define functionality. Only one of the five agencies—VA—fully 

addressed this policy component. VA has a policy that defines what it 

means to deliver functionality—both in terms of what constitutes an 

increment and what should be delivered at the end of an increment. 

For example, VA defines an increment as the segment of the project 

that produces, in a cycle of 6 months or less, a deliverable that can be 

used by customers in an operational environment.47

 45DHS, DHS Approach to Financial Systems Modernization, Version 1.3 (July 2012).

For the agency 

46Transportation officials noted that FAA is exempt from the Federal Acquisition 

Regulation and operates under an independent acquisition system, referred to as FAA’s 

Acquisition Management System.

47According to VA guidance, an increment deliverable will be defined in most situations as 

a new or enhanced IT capability used by one or more customers in production. For some 

high-risk projects, delivery of a prototype or pilot may be an acceptable increment 

deliverable. VA’s guidance also states that, for some complex systems where field 

deployment is resource intensive, the increment deliverable may be defined as the first 

production deployment.
Page 16 GAO-14-361 Incremental Development 

that partially addressed the component—Defense—although it has 

defined functionality for purposes of its acquisition framework, it has 

not defined the functionality that its IT budget submission policy 

encourages projects to deliver every 12-18 months. The department 

stated that it is working with OMB to define this term. Lastly, three 

agencies—HHS, DHS, and Transportation—had not defined 

functionality in terms of what they expected projects to deliver at the 

end of a development cycle. Officials representing these agencies’ 

respective Office of the CIO acknowledged that they have not defined 

functionality. These officials told us that they would update their 

policies to define the term, but officials from HHS and Transportation

did not provide a time frame for doing so. DHS officials stated that, 

although they did not have a definitive timeframe, they hoped to 

finalize the policy in 2014. Until the agencies define this term, 

investments may create definitions that are inconsistent with the intent 

of OMB’s policy.

• Define a process for enforcing compliance. Only one of the five 

agencies—VA—fully addressed this policy component by defining 

processes for ensuring that increments are structured to deliver 

functionality every 6 months or less and for reviewing projects that fall 

behind schedule. In particular, VA’s policy requires the agency to hold 

a TechStat session when any increment delivery date has been or will 

be missed. Two agencies partially addressed this component—

Defense and HHS—because, although they established processes for 

ensuring that IT is acquired incrementally, these processes do not (1) 

require enforcement of incremental development within the specific 

time frames consistent with OMB guidance (12-18 months for 

Defense and 3-6 months for HHS) or (2) include using TechStat 

processes to identify investments that are not being acquired 

incrementally. Finally, two agencies—DHS and Transportation—have 

not established processes for enforcing compliance with their 

incremental development policies. Officials from their respective 

Office of the CIO told us that they are updating their policies to 

address this issue. Transportation officials representing the Office of 

the CIO stated that it would update its policy later this year; DHS 

Office of the CIO officials stated that, although they did not have a 

definitive timeframe, they hoped to finalize their policy in 2014.

Agencies cited several underlying reasons that contributed to these 

weaknesses: (1) they were not always aware of OMB guidance, (2) they 

did not believe that the guidance was realistic, and (3) they said the 

guidance was not always clear. Regarding agency awareness of the 

guidance, since the 2010 IT Reform Plan, OMB has communicated 
Page 17 GAO-14-361 Incremental Development 

changes to incremental development requirements, such as the change 

from 12 to 6 months, through budget guidance.48

With respect to whether OMB’s guidance is realistic, officials from 

Defense, HHS, DHS, and Transportation explained that they do not want 

to require all of their projects to deliver functionality every 6 months 

because it may not be reasonable for all investments to do so. Defense, 

DHS, and Transportation officials said that delivering every 12 months, as 

advocated in OMB’s IT Reform Plan, is more reasonable.

However, selected 

agency officials said they were not always aware of this guidance. For 

example, DHS Office of the CIO officials told us that did not know about 

OMB’s requirement to deliver functionality every 6 months. Additionally, 

Transportation officials representing the Office of the CIO were not aware 

that in 2012 OMB had changed its guidance from recommending to 

requiring that projects deliver in 6-month cycles.

49

Lastly, two key components of OMB’s guidance are not clear. First, in 

revising Circular A-130 in 2000, OMB did not identify the minimum 

According to 

Defense officials from the Office of the CIO, 12-18 month incremental 

development is better aligned with the acquisition framework that many of 

its IT investments have used. DHS and Transportation Office of the CIO 

officials stated that, depending on program size, scope, complexity, 

budget, schedule, and expertise, it may be more reasonable to deliver 

functionality every 12 months. As discussed later in this report, we agree 

that OMB’s requirement to deliver functionality every 6 months is 

unrealistic. OMB staff members from the Office of E-Government and 

Information Technology noted that it will take time for agencies to 

embrace delivering functionality every 6 months because of the perceived 

risk of adopting such approaches. Those staff members explained that 

agencies, such as Defense, have been using the waterfall development 

method for many years, and they perceive less risk in continuing with that 

method than with changing to a method that produces functionality more 

rapidly.

 48OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012); and Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management 

of Information Technology Capital Assets (2011).

49Officials from HHS explained that they are in the process of updating their IT 

governance guidance and, as part of this process, will consider whether the agency needs 

to make any changes to its incremental development policies.
Page 18 GAO-14-361 Incremental Development 

requirements of what the agencies’ policies are to include and did not 

specify when the policies are to be completed. Although OMB issued later 

guidance on incremental development, it has not yet specified what 

agencies’ incremental development policies are to include. Second, 

OMB’s guidance did not provide a complete definition of the functionality 

it expects to be delivered every 6 months. According to staff from the 

Office of E-Government, OMB intends for agencies to deliver functionality 

that can be tested by business users; nevertheless, they noted that they 

left this definition out of their guidance so that agencies could develop a 

definition that would be flexible enough to meet their needs. However, in 

the absence of further guidance from OMB, agencies may continue to not 

define this term or may create definitions that are inconsistent with the 

intent of OMB’s policy. For example, HHS officials from the Office of the 

CIO told us that the completion of requirements documents could meet 

OMB’s definition of delivering functionality. Additionally, an FAA Office of 

the CIO official explained that some investments have classified the 

delivery of requirements documentation as functionality. These two 

examples are not consistent with OMB’s intent since they do not deliver 

functionality that can be tested, but instead only plan to deliver project 

documentation.

Until OMB explicitly issues realistic and clear guidance and Defense, 

HHS, DHS, and Transportation address the identified weaknesses in their 

incremental development policies, it will be difficult to deliver project 

functionality more rapidly, measure how often projects are delivering 

functionality, and enforce compliance with the delivery time frames called 

for in their policies.
Page 19 GAO-14-361 Incremental Development 

In its 2010 IT Reform Plan, OMB called for IT programs to deliver 

functionality at least every 12 months.50 Subsequently, OMB has made 

this requirement more stringent in that it now requires projects associated 

with major IT investments to deliver functionality every 6 months.51

The majority of the selected investments we reviewed did not plan to 

deliver functionality every 6 months. Specifically, only 23 of the selected 

89 investments had one or more projects that, when taken collectively, 

planned to deliver functionality every 6 months. To VA’s credit, all six of 

the department’s selected investments planned to deliver functionality 

every 6 months. The other agencies varied in the extent to which they 

met the standards established by OMB’s guidance. Table 2 shows how 

many of the selected investments at each agency planned on delivering 

functionality every 6 months during fiscal years 2013 and 2014.

Table 2: Number and Percentage of Selected Investments That Planned to Deliver 

Functionality Every 6 Months, by Agency, for Fiscal Years 2013 and 2014

Agency

Total number of 

selected 

investments

Number of 

investments that 

planned to deliver 

functionality every 

6 months 

Percentage of 

investments that 

planned to deliver 

functionality every 

6 months

Defense 37 1 3%

HHS 14 9 64%

DHS 12 2 17%

Transportation 20 5 25%

VA 6 6 100%

Totals 89 23 26%

Source: GAO analysis of agency data.

The variety of life-cycle cost estimates for these investments shows that 

incremental development can be applied to a wide variety of investment 

scopes. Specifically, of the 23 investments that planned to deliver 

 50OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010).

51OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012); and Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management 

of Information Technology Capital Assets (2011).

Most Selected 

Investments Do Not 

Plan to Deliver 

Functionality Every 6 

or 12 Months
Page 20 GAO-14-361 Incremental Development 

functionality in 6-month cycles, 9 had cost estimates that were less than 

$250 million, 4 had estimates between $250 and $575 million, 5 had 

estimates between $575 million and $2 billion, and 5 had estimates 

greater than $2 billion.

Twenty-seven of the 89 investments in our review (30 percent) reported 

using an Agile development methodology for one or more of their 

projects. Of those 27 investments, 14 (52 percent) planned to deliver 

functionality every 6 months.52 The other 13 investments using Agile did 

not plan on delivering functionality as frequently as OMB guidance 

requires. We have previously found that Agile projects typically produce 

working functionality every 1 to 8 weeks;53

Agency officials cited three types of investments for which it may not 

always be practical or necessary to expect functionality to be delivered in 

6-month cycles: (1) investments in life-cycle phases other than acquisition 

(i.e., planning and budgeting, management in-use, and disposition);

as such, it appears that these 

investments may not be properly implementing an Agile development 

methodology.

54

• Life-cycle phases other than acquisition. Officials from Defense, HHS, 

DHS, and Transportation stated that it is not reasonable to expect 

investments to deliver functionality every 6 months when their 

(2) 

investments intended to develop IT infrastructure; and (3) research and 

development investments.

 52In contrast, of the 62 investments not using Agile, only 9 (15 percent) planned to deliver 

functionality every 6 months.

53GAO-12-681.

54During the planning and budgeting phase, the agency performs the steps needed to 

justify the new investment and identify the amount of funding needed to deliver the IT 

asset. The acquisition phase begins after the agency has received funding from Congress 

for a segment, module, or the entire asset, and ends when the asset is delivered and fully 

operational. The management in-use phase begins after the completion of the acquisition 

phase and involves the continuous monitoring of capital assets to ensure that they are 

maintained at the right size, cost, and condition to support agency mission and objectives. 

The disposition phase involves the disposal of the capital asset. OMB defines these terms 

in its Capital Programming Guide (July 2012).
Page 21 GAO-14-361 Incremental Development 

investments’ projects are not in the acquisition phase.55 Specifically, 

24 investments did not have projects in the acquisition stage. Of those 

24 investments, 22 did not plan to deliver functionality in 6-month 

cycles (10 from Defense, 3 from HHS, 1 from DHS, and 8 from 

Transportation), and 2 investments did plan to do so (2 from HHS). 

For the 2 investments that planned to deliver functionality every 6 

months, both had at least one project in the management in-use 

phase, meaning that at least one of the investments’ projects were 

beyond the planning and development stages and were being used to 

support agency operations.

• Infrastructure investments. Officials from Defense, DHS, and 

Transportation explained that not all infrastructure investments can be 

expected to deliver functionality every 6 months. Specifically, 21 

investments provide infrastructure, such as IT security, office 

automation, and telecommunications. For example, officials 

representing two of the DHS investments explained that, prior to 

deploying functionality, they need to acquire real estate, conduct 

environmental assessments, and perform construction work, such as 

digging trenches, burying cables, and building facilities. Of the 21 

investments, 20 did not plan to deliver functionality every 6 months 

(17 from Defense, 1 from HHS, and 2 from DHS); however, 1 

investment did plan to do so (1 from Defense).

• Research and development investments. Officials from FAA’s Office 

of the CIO explained that FAA’s research and development 

investments are not intended to deliver functionality. Those officials 

stated that, before the agency approves a technology for further 

development, it performs research and development to ensure that 

the technology meets safety standards. If those standards are met, 

FAA creates a new investment aimed at deploying that technology. 

Consistent with this, none of FAA’s six research and development 

investments planned to deliver any functionality during fiscal years 

2013 and 2014.

These concerns have merit. For example, with respect to investments in 

life-cycle phases other than acquisition, at the outset of a new investment, 

 55VA’s guidance on incremental development addresses this issue: its projects are always 

in one of six life-cycle stages (new start, planning, active, closed, provisioning, and 

paused) and are only required to deliver functionality every 6 months while in the active 

stage. 
Page 22 GAO-14-361 Incremental Development 

an agency may need more than 6 months to, among other things, define 

high-level requirements and find a contractor to help develop the system. 

In addition, it may not be necessary for an investment to continue 

delivering new functionality when all planned functionality has been fully 

deployed.

Regarding infrastructure investments, it may not be practical or costeffective for an investment to refresh fully functioning hardware every 6 

months. Additionally, it may not be feasible for an agency to build a 

physical facility (e.g., data center) within 6 months. Further, for research 

and development investments, industry practices56 and our work on best 

practices57

Although OMB requires all investments to deliver functionality every 6 

months, an OMB staff member from the Office of E-Government 

explained that not all investments will be able to meet this goal. Instead, 

that staff member said that about half of the federal government’s major 

IT investments will deliver functionality in 6 months or less, and the other 

half will have longer development cycles. However, OMB’s guidance does 

not make this distinction. As a result, agencies may be confused about 

whether OMB’s incremental development guidance applies to all 

investments.

support FAA’s efforts to thoroughly validate the feasibility and 

cost-effectiveness of new technology prior to making significant 

investments.

If these three types of investments, which account for 40 of the selected 

89 investments, are not considered,58

 56SEI, CMMI-ACQ, Version 1.3 (November 2010).

29 of the remaining 49 investments 

did not plan to deliver functionality in 6-month cycles. Table 3 shows, after 

removing the three types of investments discussed above, how many of 

the selected investments at each agency planned to deliver functionality 

every 6 months during fiscal years 2013 and 2014.

57GAO, Information Technology Investment Management: A Framework for Assessing 

and Improving Process Maturity, version 1.1, GAO-04-394G (March 2004).

58Five investments are both (1) in life-cycle phases other than acquisition and (2) intended 

to develop IT infrastructure, and 6 investments are both (1) in life-cycle phases other than 

acquisition and (2) research and development investments. As such, the total number of 

investments not considered is 40 rather than 51.
Page 23 GAO-14-361 Incremental Development 

Table 3: Number and Percentage of Remaining Selected Investments That Planned 

to Deliver Functionality Every 6 Months, by Agency, for Fiscal Years 2013 and 2014

Agency

Total number 

of remaining 

investments

Number of remaining 

investments that 

planned to deliver 

functionality every 6 

months

Percentage of 

remaining 

investments that 

planned to deliver 

functionality every 6 

months

Defense 14 0 0%

HHS 8 7 88%

DHS 9 2 22%

Transportation 12 5 42%

VA 6 6 100%

Totals 49 20 41%

Source: GAO analysis of agency data.

Considering agencies’ concerns about delivering functionality every 6 

months for the three types of investments discussed above and OMB’s 

own expectations that many investments will not meet this goal, it is 

unclear whether this is the most appropriate governmentwide goal, and it 

raises the question of whether OMB should consider a longer time frame, 

such as 12 months, as called for in OMB’s IT Reform Plan. However, 

even using the time frame of 12 months as the target, less than half of the 

selected investments planned to deliver functionality in 12-month cycles. 

Specifically, 41 of the 89 selected investments planned to deliver 

functionality every 12 months during fiscal years 2013 and 2014 and 48 

did not. Most notably, the preponderance of Defense and Transportation

investments (70 percent for Defense and 65 percent for Transportation) 

did not plan to deliver functionality every 12 months. Table 4 shows how 

many of the selected investments at each agency planned on delivering 

functionality every 12 months during fiscal years 2013 and 2014.
Page 24 GAO-14-361 Incremental Development 

Table 4: Number and Percentage of Selected Investments That Planned to Deliver 

Functionality Every 12 Months, by Agency, for Fiscal Years 2013 and 2014

Agency

Total number 

of selected 

investments

Number of 

investments that 

planned to deliver 

functionality every 12 

months 

Percentage of 

investments that 

planned to deliver 

functionality every 12 

months

Defense 37 11 30%

HHS 14 11 79%

DHS 12 6 50%

Transportation 20 7 35%

VA 6 6 100%

Totals 89 41 46%

Source: GAO analysis of agency data.

The previously discussed weaknesses in agency policies have permitted 

the inconsistent implementation of incremental development approaches. 

OMB staff members from the Office of E-Government acknowledged that 

inconsistent implementation of OMB’s guidance can be at least partially 

attributed to challenges in ensuring that agencies develop consistent 

definitions of and approaches to incremental development.

Although OMB has led the government’s recent effort to improve 

incremental development, it has not completed a key commitment aimed 

at improving incremental development—namely, OMB has rarely used 

the TechStat process to turn around or cancel investments that are not 

using incremental development. As previously mentioned, TechStat 

sessions are intended to terminate or turn around IT investments that are 

failing or are not producing results. Additionally, many failed IT 

investments59 have not used incremental development approaches. 

Therefore, OMB could use TechStat sessions as a powerful tool to turn 

around investments that are not being acquired incrementally. However, 

OMB staff members from the Office of E-Government said that OMB has 

only held one such TechStat.60

 59GAO-13-796T.

60According to OMB staff members from the Office of E-Government, this investment was 

Transportation’s Federal Motor Carrier Safety Administration Application and Information 

Services Modernization. OMB reported holding the TechStat session in August 2012.
Page 25 GAO-14-361 Incremental Development 

OMB staff from the Office of E-Government explained that, in order to 

select investments in need of a TechStat session, agency exhibit 300 

data are reviewed for evidence of poor performance. However, the 

usefulness of the exhibit 300 data for the purpose of identifying whether 

investments are using incremental approaches is limited. For example, 

OMB does not require agencies to explicitly identify whether their 

investments are using incremental approaches. Additionally, of the 89 

selected investments, 34 had activities in their exhibit 300 submissions 

that were inaccurately classified as delivering functionality (9 from 

Defense, 8 from HHS, 5 from DHS, 9 from Transportation, and 3 from 

VA). For example, one Defense investment indicated that awarding a 

contract constituted a delivery of functionality. OMB staff from the Office 

of E-Government acknowledged that the exhibit 300 data are not as 

helpful as possible in addressing incremental development. 

Consequently, for its TechStat reviews, OMB’s insight into investments’ 

use of incremental development approaches is limited. Officials from 

Defense, HHS, DHS, and Transportation attributed the problem to a lack 

of guidance from OMB on what is to be delivered every 6 months.61

Nevertheless, officials from Defense, DHS, and VA stated that they would 

properly classify activities in future exhibit 300 submissions.62

Without better implementation of incremental development approaches 

and identification of investments using incremental development, IT 

expenditures will likely continue to produce disappointing results—

including large cost overruns, long schedule delays, and questionable 

mission-related achievements. Further, without useful information, 

including whether investments are following an incremental approach, on 

projects associated with major IT investments, OMB does not have the 

However, 

without additional guidance from OMB, agencies may continue to 

improperly classify activities. Additionally, as previously mentioned, four 

of the five selected agencies—Defense, HHS, DHS, and Transportation—

have not updated their TechStat policies to include identifying 

investments that are not being acquired incrementally.

 61A VA official from the Office of the CIO attributed the problems to confusion about how 

the 6-month requirement applies to projects that deploy functionality to sites across the 

country.

62HHS officials representing the Office of the CIO explained that they will address this 

issue if they see patterns of problems with exhibit 300 submissions. Transportation Office 

of the CIO officials stated that a clear definition from OMB on what functionality is to be 

delivered every 6 months is needed to address this issue.
Page 26 GAO-14-361 Incremental Development 

necessary information to oversee the extent to which projects and 

investments are implementing its guidance. 

Multiple factors were identified by the five agencies as enabling and 

inhibiting incremental development during a 6-month period. Specifically, 

eight factors were identified by three or more of the five agencies in our 

review as enabling incremental development of IT systems, and seven 

factors were identified by three or more agencies as inhibiting incremental 

development. The enabling factor identified by all of five of the agencies 

was active engagement of program officials with stakeholders. The 

inhibiting factors identified by all five agencies were (1) the lack of 

sufficient, timely funding; (2) program characteristics that made rapid 

incremental development infeasible; and (3) the lack of stable, prioritized 

requirements.63

Eight factors were identified by three or more of the five agencies in our 

review as contributing to the successful development of functionality in 6-

month cycles. All five of the agencies in our review cited active program 

engagement with stakeholders. Table 5 shows the distribution of the eight 

factors among the agencies, and examples of how the agencies 

implemented them are discussed following the table.

 63Defense also reported similar enabling and inhibiting factors for the 12-18 month period 

called for in its guidance.

Multiple Factors Were 

Commonly Identified 

as Enabling and 

Inhibiting Incremental 

Development

Eight Factors Were 

Commonly Identified as 

Enabling Incremental 

Development during a 6-

month Period
Page 27 GAO-14-361 Incremental Development 

Table 5: Commonly Identified Factors Enabling Incremental Development during a 6-month Period

Enabling factor for incremental development Defense HHS DHS Transportation VA

1 Program officials actively engaged with stakeholders. X X X X X

2 Programs used an Agile development methodology. X X X X

3 Programs successfully prioritized, managed, and tested requirements. X X X X

4 Staff had the necessary skills and experience. X X X X

5 Programs successfully implemented cost and schedule estimating best 

practices.

X X X X

6 Officials used various contracting strategies to increase flexibility and 

improve contractor oversight.

X X X X

7 Staff used key technologies to accelerate development work. X X X

8 Programs successfully implemented risk management best practices. X X X

Source: GAO analysis of agency data.

Note: Where factors were cited by the same number of agencies, the factors are listed in descending 

order from the greatest amount of investments that cited them to the least amount.

• Program officials actively engaged with stakeholders. Officials from all 

five agencies explained that active engagement with program 

stakeholders—individuals or groups with an interest in the success of 

the investment—was a factor that enabled the development of 

functionality in 6-month cycles. For example, officials from one of the 

HHS investments that we reviewed stated that having strong 

communication between the business program, the IT program, and 

contractors enabled projects to move forward on schedule in a 

cohesive manner with clear goals and objectives.

• Programs used an Agile development methodology. Officials from four 

of the five agencies indicated that the use of an Agile development 

methodology helped them to deliver functionality every 6 months. For 

example, Defense officials explained that the use of Agile 

development processes allowed software to be broken into smaller 

releases that were easily achieved, tested, and fielded. Additionally, 

VA officials stated that the agency has embraced Agile development, 

which has helped the department deliver functionality more quickly. 

Those officials also noted that merely forcing investments to use 6-

month waterfall iterations would not have resulted in the same 

success.

• Programs successfully prioritized, managed, and tested requirements. 

Officials from four of the five agencies identified implementation of 

requirements management practices—including requirements 

prioritization, management, and testing—as a factor that enabled the 
Page 28 GAO-14-361 Incremental Development 

agencies to deliver functionality every 6 months. For example, 

Transportation officials explained that the primary factor that enabled 

incremental development has been obtaining clear requirements. 

Further, HHS officials stated that the use of a prioritized product 

backlog64 has helped teams make decisions about which 

requirements should be allocated to future releases.

• Staff had the necessary skills and experience. Officials from four of 

the five agencies stated that consistent and stable program staff with 

experience and expertise allowed them to deliver functionality 

frequently. For example, DHS officials representing one of the 

selected investments indicated that having skilled program managers 

has a large impact on the success of a program. In addition, VA 

officials reported that their ability to retain key personnel with both 

functional and technical expertise has enabled them to deliver highquality functionality rapidly.

• Programs successfully implemented cost and schedule estimating 

best practices. Officials from four of the five agencies cited the 

implementation of cost and schedule estimating practices as enabling 

the frequent delivery of functionality. For example, Defense officials 

told us that development of realistic cost estimates and 

comprehensive program schedules helped programs to deliver 

functionality while meeting established performance goals. Further, 

Transportation officials stated that improved guidance and training on 

cost estimating and scheduling helped them to deliver functionality in 

6-month cycles.

• Officials used various contracting strategies to increase flexibility and 

improve contractor oversight. Officials from four of the five agencies 

cited the use of specific contracting strategies. For example, DHS 

officials indicated that they worked with contractors to modify their 

contracts from the traditional waterfall approach to a structure that 

allows for Agile development. In addition, officials from VA stated that 

the use of performance-based acquisitions assisted them in 

monitoring contractor progress towards achieving actual results 

against planned objectives.

 64The backlog is a list of customer requirements to be addressed by the functionality.
Page 29 GAO-14-361 Incremental Development 

• Staff used key technologies to accelerate development work. Officials 

from three of the five agencies indicated that the use of key 

technologies enabled them to deliver functionality more quickly. For 

example, HHS officials explained that having an established cloud 

environment has enabled them to reduce deployment time for 

releases, while also providing the needed flexibility to meet their 

customers’ changing needs. Additionally, DHS officials explained that 

projects, especially those using an Agile development methodology, 

have saved time using automated testing tools.

• Programs successfully implemented risk management best practices. 

Officials from three agencies explained that the successful 

implementation of risk management practices helped them to deliver 

functionality more rapidly. For example, officials from one of the 

selected VA investments told us that the organization’s risk 

management process allowed teams to quickly escalate risks to 

senior leadership so that they could be managed before their impacts 

were fully realized.

Many of the factors that the five agencies cited are consistent with our 

2011 work on factors critical to successful IT acquisitions and our 2012 

work on effective practices in implementing Agile software development.65

Additionally, the eight commonly identified factors that enable the delivery 

of functionality in 6 months are consistent with OMB’s IT Reform Plan.

In particular, our work in both areas discussed the importance of active 

engagement with program stakeholders, such as actively engaging with 

stakeholders to obtain customer feedback, and effectively managing 

requirements.

66

 65

In particular, as previously mentioned, one high-level objective of the 

plan—effectively managing large-scale IT programs—aims to improve 

areas that impact the success rates of large IT programs by, among other 

things, enabling programs to use incremental development approaches. 

As part of this high-level objective, the plan addresses the importance of 

actively engaging with stakeholders, ensuring that program management 

professionals have proper skills and experience, and developing new 

GAO-12-681 and GAO-12-7.

66OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010).
Page 30 GAO-14-361 Incremental Development 

contracting strategies that more effectively support incremental 

development.

Seven factors were identified by three or more of the five agencies as 

inhibiting the development of IT functionality every 6 months. The factors 

most commonly cited include (1) programs did not receive sufficient 

funding or receiving funding later than needed, (2) program 

characteristics made rapid delivery of functionality infeasible or 

impractical, (3) programs did not have stable and prioritized requirements. 

Table 6 shows the distribution of the seven factors among the agencies, 

and examples of how the factors impacted the agencies are discussed 

following the table.

Table 6: Commonly Identified Factors Inhibiting Incremental Development during a 6-month Period

Inhibiting factor for incremental development Defense HHS DHS Transportation VA

1 Programs did not receive sufficient funding or received funding later than 

needed.

X X X X X

2 Program characteristics made rapid delivery of functionality infeasible or 

impractical.

X X X X X

3 Programs did not have stable, prioritized requirements. X X X X X

4 Development work was slowed by inefficient governance and oversight 

processes.

X X X

5 Development schedules were impeded by procurement delays. X X X

6 Program staff were overutilized or lacked the necessary skills and experience. X X X

7 Incremental development was impeded by select technologies. X X X

Source: GAO analysis of agency data.

Note: Where factors were cited by the same number of agencies, the factors are listed in descending 

order from the greatest amount of investments that cited them to the least amount.

• Programs did not receive sufficient funding or received funding later 

than needed. Officials from all five departments cited insufficient 

funding, such as reductions caused by the fiscal year 2013 sequester, 

or receiving funding later than needed because of continuing 

resolutions. For example, Defense officials representing one of the 

investments we reviewed stated that furloughs brought on by the 2013 

sequester significantly impacted program schedules, both in terms of 

lost work days and the inability to coordinate integration between 

software providers, resulting in overall inefficiencies. In addition, 

several FAA officials explained that the delivery of planned 

functionality was adversely affected by the uncertainty brought about 

by the 2013 sequester, and that future funding instability has impacted 

the agency’s ability to plan when functionality is to be delivered.

Seven Factors Were 

Commonly Identified as 

Inhibiting Incremental 

Development during a 6-

month Period
Page 31 GAO-14-361 Incremental Development 

• Program characteristics made rapid delivery of functionality infeasible 

or impractical. Officials from all five agencies indicated that some of 

their programs have certain characteristics—such as the deployment 

of new physical infrastructure, human health and safety concerns, or 

external dependencies—that make the delivery of functionality in 6-

month time frames infeasible or impractical. For example, DHS 

officials explained that their infrastructure projects cannot deliver 

functionality until all key activities (e.g., land acquisition, 

environmental assessments, site preparation, and construction) have 

been completed and the new infrastructure has been fully deployed, 

tested, and accepted. Additionally, Transportation officials reported 

that air traffic control systems require years of development and 

testing in order to ensure that the systems do not compromise 

airspace security.

• Programs did not have stable, prioritized requirements. Officials from 

all five agencies stated that not having complete requirements at the 

beginning of their investments’ development or having changes made 

to requirements and their relative priorities during development was a 

factor that negatively affected their programs’ delivery of incremental 

functionality in 6-month periods. For example, HHS officials 

representing one of the selected investments explained that that final 

rules detailing eligibility for a medical program were not completed 

until late in the development cycle; consequently, the program could 

not define the corresponding business requirements in time to meet 

the development schedule. Further, VA officials stated that schedules 

are disrupted when stakeholders identify new program requirements 

that must be delivered before others.

• Development work was slowed by inefficient governance and 

oversight processes. Officials from three of the five agencies cited 

inefficient governance and oversight processes. For example, DHS 

officials representing the Office of the CIO stated that the current DHS 

governance model was not conducive to frequent delivery of 

functionality. To illustrate, those officials noted that it can take up to 2 

months to schedule a meeting with DHS review boards prior to 

releasing functionality. However, a DHS official from Program 

Accountability and Risk Management disagreed with this statement, 

explaining that DHS’s acquisition review boards perform reviews very 

quickly, and that any delays in completing these reviews are 

attributable to investments being unprepared. Further, DHS Office of 

the CIO officials suggested that governance over programs using an 

Agile development methodology should be performed at the lowest 

practicable level of the organization. In addition, VA officials explained 
Page 32 GAO-14-361 Incremental Development 

that, although the periodic approvals required of various department 

groups are useful, these approvals can slow the testing and release 

processes and could use further streamlining.

• Development schedules were impeded by procurement delays. 

Officials from three of the five agencies explained that procurement 

delays—such as delays in getting contracts awarded or approving 

contract modifications—contributed to difficulties in delivering 

incremental functionality. For example, DHS officials explained the 

process of planning for an acquisition, developing solicitations, 

selecting contractors, and, in some cases, addressing protests, are 

not conducive to delivering functionality in 6-month cycles.

• Program staff were overutilized or lacked the necessary skills and 

experience. Officials from three of the five agencies indicated that 

they did not have enough program staff with the expertise and 

experience necessary to deliver functionality every 6 months. For 

example, officials representing one of the HHS investments stated 

that the loss of key personnel and the lack of staff with knowledge and 

skill in key disciplines, such as incremental development, have 

negatively impacted the agency’s ability to deliver functionality.

• Incremental development was impeded by select technologies. 

Officials from three of the five agencies in our review explained that 

the software and tools they selected for their programs either 

introduced delays in development or were ultimately not usable by the 

program. Transportation officials representing one of the selected 

investments reported that their chosen technology added a new level 

of complexity to storage and data processing that required all 

development and testing to be completed before the implementation 

could occur. As a result, those officials told us that all functionality had 

to be deployed in one release, instead of being spread out in stages.

Many of the factors that the five agencies identified as inhibiting delivery 

of functionality in 6-month increments were consistent with our work on 

the challenges we identified in the application of Agile software 

development methods.67

 67

For example, we noted that one challenge in 

deploying software developed using an Agile development method is that 

traditional governance and oversight activities were difficult to execute 

GAO-12-681.
Page 33 GAO-14-361 Incremental Development 

within an iterative time frame. In particular, one agency official stated that 

completing the necessary reviews within the short, fixed time frame of an 

Agile iteration was difficult because reviewers followed a slower waterfall 

schedule with reviews that could take months to perform after the 

completion of an iteration. This caused delays for iterations that needed 

such reviews within the few weeks of the iteration. We also noted that 

procurement practices sometimes delay Agile development.

Further, OMB’s IT Reform Plan also identified some of the same 

problems that inhibit incremental development and described solutions to 

address them.68

The factors identified in this report as enabling and inhibiting incremental 

development could help agencies address the challenges they face in 

incrementally acquiring IT.

In particular, as previously mentioned, one item of the 

plan was to improve funding of programs by working with Congress to 

create IT budget models that promote IT budget flexibility (such as 

multiyear budgets or revolving funds). In addition, the plan addresses the 

importance of streamlining IT governance by strengthening the quality 

and timing of oversight activities.

Given the enormous size of the federal government’s investment in IT 

and the often disappointing results from IT development efforts—many of 

which are attributable to the use of a “big bang” approach—finding ways 

to improve the quality and timeliness of agencies’ investments is 

important. Congress, OMB, and our work support the use of incremental 

development practices. However, although the selected agencies have 

developed policies that address incremental development, most of the 

agencies’ policies have significant weaknesses. With the exception of VA, 

these policies do not fully define functionality, and do not have a complete 

process for ensuring that the agencies’ investments are developed 

incrementally, including the use of TechStat sessions to enforce 

compliance with incremental development policies. In the absence of 

such policies, agencies continue to run the risk of failing to deliver major 

investments in a cost-effective and efficient manner.

 68OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010). 

Conclusions
Page 34 GAO-14-361 Incremental Development 

Regarding implementation of incremental development, slightly more than 

one-fourth of selected investments planned to deliver functionality every 6 

months—and less than one-half planned to do so every 12 months. Thus, 

delivering functionality every 6 months is not an appropriate requirement 

for all agencies given current performance. Requiring the delivery of 

functionality every 12 months, consistent with OMB’s IT Reform Plan, 

would be an appropriate starting point and be a substantial improvement. 

Further, since there are three types of investments for which it may not 

always be practical or necessary to expect functionality to be delivered in 

6-month cycles, this raises questions about whether OMB’s requirement 

for shorter delivery cycles should be applied to all investments or whether 

investments should be allowed the latitude to determine a more 

appropriate time frame.

The lack of progress in updating policies and implementing OMB’s 

guidance was enabled by weaknesses in OMB’s guidance. In the 

absence of agency and OMB use of TechStat sessions to ensure 

compliance with incremental development policy, investments will 

continue to be at risk of not delivering promised capabilities on time and 

within budget. Until OMB clearly and explicitly disseminates guidance with 

realistic goals and clear expectations, and agencies update their policies 

to reflect this guidance, agencies may not consistently adopt incremental 

development approaches, and IT expenditures will continue to produce 

disappointing results—including sizable cost overruns and schedule 

slippages, and questionable progress in meeting mission goals and 

outcomes. Additionally, without useful information on how often projects 

are delivering functionality, it will be difficult for OMB to identify 

investments that are not implementing its guidance. Further, 

dissemination of the factors identified in this report as enabling and 

inhibiting incremental development may help federal agencies address 

the challenges they face in acquiring IT investments faster and more 

efficiently.

We recommend that the Director of the Office of Management and 

Budget direct the Federal Chief Information Officer to take the following 

two actions.

• Update, and clearly and explicitly issue incremental development 

guidance that addresses the following three components:

• requires projects associated with major IT investments to deliver 

incremental functionality at least every 12 months, with the 

exception of the three types of investments identified in this report;

Recommendations for 

Executive Action
Page 35 GAO-14-361 Incremental Development 

• specifies how agencies are to define the project functionality that 

is to be delivered; and

• requires agencies to define a process for enforcing compliance 

with incremental functionality delivery, such as the use of 

TechStat sessions.

• Require agencies to clearly identify on exhibit 300 submissions 

whether, for each project, functionality will be delivered within the time 

frames called for by this incremental development guidance, and to 

provide justification for projects that do not plan to do so.

We further recommend that the Secretaries of Defense, Health and 

Human Services, Homeland Security, and Transportation take the 

following two actions:

• modify, finalize, and implement their agencies’ policies governing 

incremental development to ensure that those policies comply with 

OMB’s guidance, once that guidance is made available; and

• when updating their policies, consider the factors identified in this 

report as enabling and inhibiting incremental development.

We also recommend that the Secretary of Veterans Affairs consider 

incorporating the factors identified in this report as enabling and inhibiting 

incremental development in the department’s related policy.

We received comments on a draft of this report from OMB and the five 

agencies in our review. OMB agreed with one recommendation and 

partially disagreed with the other; Defense generally concurred with the 

report; HHS neither agreed nor disagreed with the report’s 

recommendations; DHS agreed with our recommendations; 

Transportation did not agree with the recommendations in that it did not 

believe the department should be dependent on OMB first taking action; 

and VA generally agreed with the report’s conclusions and concurred with 

our recommendation. Each agency’s comments that we received are 

discussed in more detail below.

• In comments provided via e-mail on April 15, 2014, staff in OMB’s 

Office of General Counsel, on behalf of OMB, stated that the agency

agreed with one of our recommendations and partially disagreed with 

the other. Specifically, OMB agreed with our recommendation to 

Agency Comments 

and Our Evaluation
Page 36 GAO-14-361 Incremental Development 

require agencies to clearly identify on exhibit 300 submissions 

whether functionality will be delivered within the time frames called for 

by this incremental development guidance.

OMB stated that it agreed with our recommendation to update and 

issue incremental development guidance, but did not agree that this 

guidance should require major IT investments to deliver incremental 

functionality at least every 12 months. OMB explained that changing 

the requirement from 6 to 12 months would reduce the emphasis on 

incremental development that it has been advocating. OMB also 

noted that it believes requiring investments to deliver functionality 

every 6 months is an appropriate governmentwide goal and said that 

updating and clarifying its guidance on incremental development will 

make it easier for agencies to meet this target. However, as we state 

in this report, slightly more than one-fourth of selected investments 

planned to deliver functionality every 6 months—and less than onehalf planned to do so every 12 months. Additionally, there are three 

types of investments for which it may not always be practical or 

necessary to expect functionality to be delivered in 6-month cycles.

Thus, we continue to believe that delivering functionality every 6 

months is not an appropriate requirement for all agencies and that 

requiring the delivery of functionality every 12 months, consistent with 

OMB’s IT Reform Plan, is a more appropriate starting point. We 

therefore maintain that OMB should require projects associated with 

major IT investments to deliver functionality at least every 12 months.

• In written comments, Defense stated that it generally concurred with 

the report and outlined planned actions to address the 

recommendations. However, Defense explained that many of its IT 

investments are consistent with the three types of investments for 

which it may not always be practical or necessary to expect 

functionality to be delivered in 6-month cycles or have other 

exceptional circumstances. Given these issues, Defense stated that 

requiring the delivery of functionality every 12 months is not a useful 

management constraint. However, as we state in our report, many 

failed projects have been broadly scoped in that they aim to deliver 

their capabilities several years after initiation. Additionally, the 

Defense Science Board reported that Defense’s acquisition process 

for IT systems was too long, was ineffective, and did not 

accommodate the rapid evolution of IT. In order to resolve these 

issues, we continue to believe that investments and their projects 

should deliver functionality more frequently. In addition, the existence 

of many investments that are consistent with the three types of 
Page 37 GAO-14-361 Incremental Development 

investments identified in this report for which it may not be practical to 

deliver functionality quickly does not excuse Defense from the 

requirement of delivering functionality every 12 months. Consistent 

with our recommendation to OMB, investments and projects that meet 

these three exceptions should be exempt from the requirement, but all 

other investments and projects should aim to meet this requirement. 

Further, we expect that Defense’s actions to implement our 

recommendation to consider the factors identified in this report that 

enable and inhibit incremental development will help its investments 

and projects deliver functionality more rapidly. As such, we continue 

to believe that requiring the delivery of functionality every 12 months 

is an appropriate goal for all federal agencies, including Defense.

Defense’s comments are reprinted in appendix III. The department 

also provided technical comments, which we have incorporated in the 

report as appropriate.

• In comments provided via e-mail on April 15, 2014, an official from 

HHS’s Office of the Assistant Secretary for Legislation, on behalf of 

HHS, stated that the department had no comments.

• In written comments, DHS stated that it concurred with our 

recommendations and outlined planned actions to address the 

recommendations. DHS’s comments are reprinted in appendix IV. The 

department also provided technical comments, which we have 

incorporated in the report as appropriate.

• In comments provided via e-mail on April 15, 2014, Transportation’s 

Deputy Director of Audit Relations, on behalf of Transportation, stated 

that the department would prefer to have specific recommendations 

and deliverables so that it can achieve success in closing them. 

Specifically, the department explained that relying on another agency 

to concur with one of our recommendations before Transportation can 

take action leaves the department with the potential challenge of a 

recommendation that cannot be implemented. However, as previously 

stated, OMB agrees with our recommendation to update and issue 

incremental guidance, meaning that OMB has committed to taking the 

actions necessary to enable Transportation to begin addressing our 

recommendation. Additionally, our recommendation to consider the 

factors identified in this report as enabling and inhibiting incremental 

development when updating incremental development policies does 

not require another agency to take action before Transportation can 

implement it. Accordingly, we continue to believe that our 

recommendations are warranted and can be implemented.
Page 38 GAO-14-361 Incremental Development 

• In written comments, VA stated that it generally agreed with the 

report’s conclusions, that it concurred with our recommendation to the 

department, and that it will review its existing policy for incremental

development and consider incorporating the factors identified as 

enabling and inhibiting incremental development. VA’s comments are 

reprinted in appendix V. The department also provided technical 

comments, which we have incorporated in the report as appropriate.

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to interested 

congressional committees; the Director of the Office of Management and 

Budget; and the Secretaries of the Departments of Defense, Health and 

Human Services, Homeland Security, Transportation, and Veterans 

Affairs. In addition, the report will also be available at no charge on our 

website at http://www.gao.gov/.

If you or your staffs have any questions on matters discussed in this 

report, please contact me at (202) 512-9286 or pownerd@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this report. GAO staff who made 

major contributions to this report are listed in appendix VI.

David A. Powner

Director

Information Technology

Management Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 39 GAO-14-361 Incremental Development 

Our objectives for this review were to (1) assess whether selected 

agencies have established policies for incremental information technology 

(IT) development, (2) determine whether selected agencies are using 

incremental development approaches to manage their IT investments, 

and (3) identify the key factors that enabled and inhibited the selected 

agencies’ abilities to effectively use incremental development approaches 

to manage their IT investments.

In conducting our review, we selected five agencies and a total of 89 

investments from those agencies. To choose the agencies, we identified 

the five agencies with the largest IT budgets for development, 

modernization, and enhancement on major IT investments in fiscal years 

2013 and 2014 as reported in the Office of Management and Budget’s 

(OMB) fiscal year 2014 exhibit 53.1 Those agencies are the Departments 

of Defense (Defense), Health and Human Services (HHS), Homeland 

Security (DHS), Transportation (Transportation), and Veterans Affairs 

(VA). To choose the agencies’ investments, we identified the 98 major IT 

investments for which the selected agencies planned to spend more than 

50 percent of the investments’ fiscal year 2013 and 2014 budgets on 

development, modernization, and enhancement as reported in OMB’s 

fiscal year 2014 exhibit 53. We removed 9 investments because, after 

reporting their planned budgets for OMB’s fiscal year 2014 exhibit 53, 

these investments changed their budgets so that they no longer planned 

to spend more than 50 percent of the investments’ fiscal year 2013 and 

2014 budgets on development, modernization, and enhancement.2

 1

The purpose of the exhibit 53 is to identify all IT investments—both major and 

nonmajor—and their associated costs.

The 

final 89 investments are identified in appendix II. The investments 

selected for the review account for about 58 percent of the development, 

modernization, and enhancement spending on all federal agencies’ major 

IT investments for fiscal years 2013 and 2014 reported in OMB’s exhibit 

53 for fiscal year 2014.

2

Those investments are Defense’s Air Force Intranet Increment 2; HHS’s Federal Drug 

Administration Office of Regulatory Affairs Regulatory Business Information Services; 

DHS’s TacNet, Immigration and Custom’s Enforcement Student and Exchange Visitor 

Information System, and Federal Emergency Management Agency Infrastructure; and 

Transportation’s Federal Motor Carrier Safety Administration Inspection Services, Federal 

Motor Carrier Safety Administration Compliance Services, Federal Motor Carrier Safety 

Administration Registration Services, and Pipeline and Hazardous Materials Safety 

Administration National Pipeline Information Exchange.

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 40 GAO-14-361 Incremental Development 

To address our first objective, we reviewed OMB guidance related to the 

use of incremental development, as well as industry guidance, and 

identified three key components of incremental development that 

agencies should include in their policies. In order to identify these 

components, we reviewed OMB’s guidance3 and leading industry 

guidance on institutionalizing processes throughout an organization.4

• Require delivery of functionality every 6 months. According to the 

Software Engineering Institute’s (SEI) Capability Maturity Model® 

Integration for Acquisition (CMMI-ACQ), as part of institutionalizing a 

process, organizations should document their processes, to include 

defining standards and requirements.

Our 

analysis identified three key policy components that will help agencies 

effectively implement OMB’s requirement for incremental development.

5 According to OMB budget 

guidance, projects associated with major IT investments must deliver 

functionality every 6 months.6

• Define functionality. As previously stated, according to SEI’s CMMIACQ, as part of institutionalizing a process, organizations should 

document that process, to include defining standards and 

requirements.7

 3

OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Executive Office of the President of the United States, OMB, Fiscal Year 2014 

Analytical Perspectives: Budget of the U.S. Government, 354; OMB, Contracting 

Guidance to Support Modular Development (June 14, 2012); Guidance on Exhibits 53 and 

300—Information Technology and E-Government (2012); Guidance on Exhibit 300—

Planning, Budgeting, Acquisition, and Management of Information Technology Capital 

Assets (2011); and Management of Federal Information Resources, Circular No. A-130 

Revised.

4

Software Engineering Institute, Capability Maturity Model® Integration for Acquisition, 

Version 1.3 (November 2010).

5

SEI, CMMI-ACQ, Version 1.3 (November 2010).

6

OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012); Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management of 

Information Technology Capital Assets (2011). In 2011, OMB changed its guidance to 

recommend that projects deliver functionality every 6 months. Since 2012, OMB has 

required that projects deliver functionality in 6-month cycles.

7

SEI, CMMI-ACQ, Version 1.3 (November 2010).
Appendix I: Objectives, Scope, and 

Methodology

Page 41 GAO-14-361 Incremental Development 

• Define a process for enforcing compliance. According to SEI’s CMMIACQ, as part of institutionalizing a process, organizations should 

document that process, including management review activities (e.g., 

taking corrective action when requirements and objectives are not 

being satisfied).8 Additionally, OMB Circular A-130 requires agencies 

to ensure that investments comply with their policies.9 Further, 

according to the President’s Fiscal Year 2014 Budget, agencies are to 

use their TechStat processes to identify investments that are not 

being acquired incrementally and undertake corrective actions.10

At each selected agency, we then analyzed agency policies for 

incremental development and compared these policies to the three 

components identified above.

For each agency, each policy component was assessed as either being 

not met—the agency did not provide evidence that addressed the 

component or provided evidence that minimally addressed the 

component; partially met—the agency provided evidence that addressed 

about half or a large portion of the component; or fully met—the agency 

provided evidence that addressed the component. We also interviewed 

officials from OMB and the five selected agencies to obtain information 

about their current and future incremental development policies.

To address our second objective, we administered a data collection 

instrument to each of the selected investments about how often each 

investment planned to deliver functionality during fiscal years 2013 and 

2014. We then analyzed information obtained from data collection 

instruments describing how often the selected investments planned to 

deliver functionality. We prepopulated these instruments with data 

obtained from OMB’s fiscal year 2014 exhibit 53, as well as each 

investment’s exhibit 300 data, which describe the investments’ projects 

and how often each project plans to deliver functionality. We asked 

officials from each investment to verify the accuracy and completeness of 

 8

SEI, CMMI-ACQ, Version 1.3 (November 2010).

9

OMB, Management of Federal Information Resources, Circular No. A-130 Revised. In 

2012, OMB reaffirmed its call for agencies to develop policies on incremental 

development, but it did not specify what these policies are to include. OMB, Contracting 

Guidance to Support Modular Development (June 14, 2012).

10Executive Office of the President of the United States, OMB, Fiscal Year 2014 Analytical 

Perspectives: Budget of the U.S. Government, 354.
Appendix I: Objectives, Scope, and 

Methodology

Page 42 GAO-14-361 Incremental Development 

the data and to make corrections where needed. Because the exhibit 300 

data did not always describe the investments’ plans for delivering 

functionality in fiscal year 2014, we asked the officials to indicate for each 

project whether they planned to deliver functionality in the first half of 

fiscal year 2014 (i.e., from October 1, 2013, to March 31, 2014) and in the 

second half of fiscal year 2014 (i.e., from April 1, 2014, to September 30, 

2014). We also asked the investments to provide their life-cycle cost 

estimates and, for each project, the development methodology used and 

phase of the acquisition life cycle.

Using the information obtained through the data collection instruments, 

we determined the extent to which the selected investments and their 

projects planned to meet OMB’s guidance on incremental development. 

To assess whether investments had planned to deliver functionality every 

6 months, we determined whether the selected investments planned to 

deliver functionality in each of the following four time frames: (1) the first 

half of fiscal year 2013 (i.e., from October 1, 2012, to March 31, 2013), (2) 

the second half of fiscal year 2013 (i.e., from April 1, 2013, to September 

30, 2013), (3) the first half of fiscal year 2014, and (4) the second half of 

fiscal year 2014. To determine whether investments had planned to 

deliver functionality every 12 months, we analyzed whether the selected 

investments planned to deliver functionality in each of the following two 

time frames: (1) fiscal year 2013 and (2) fiscal year 2014. We presented 

our results to the five selected agencies and OMB and solicited their input 

and explanations for the results.

To determine the reliability of the exhibit 300 data, we performed three 

steps. First, as previously mentioned, we asked officials from each 

investment to verify the accuracy and completeness of these data and 

provide the correct information where needed. Second, we removed 

projects that were not intended to deliver functionality and activities that 

were inaccurately classified as resulting in the delivery of functionality. To 

do so, we compared the descriptions of the projects and activities with a 

definition of functionality that we developed. To develop this definition, we 
Appendix I: Objectives, Scope, and 

Methodology

Page 43 GAO-14-361 Incremental Development 

reviewed OMB11 and agency guidance,12 as well as leading practices.13

To address our third objective, as part of the previously mentioned data 

collection instrument, we asked the five selected agencies and 

investments to identify the key factors that have both enabled and 

inhibited their efforts to deliver functionality for their major IT investments 

every 6 months, consistent with OMB guidance. We also asked these 

questions of officials from each of the selected agencies’ Office of the 

Chief Information Officer (CIO). Because of the open-ended nature of our 

questions, we conducted a content analysis of the information we 

received in order to identify common factors. We then totaled the number 

of times each factor was mentioned by department and agency officials, 

choosing to report on the factors that were identified by three or more 

agencies. We then compared these factors to our recent work on critical 

factors underlying the success of major IT acquisitions,

We defined functionality as follows: the implementation of IT requirements 

that is intended to either (1) ultimately be used by one or more customers 

in production (actual deployment may occur at a later date than when the 

functionality is reported as being delivered) or (2) be a delivery of a 

prototype or pilot. We then assessed the descriptions of the activities and 

projects agencies reported in their exhibit 300 submissions against our 

definition. We presented the activities and projects that did not meet our 

definition to the selected agencies and solicited their input and 

explanations. Third, where there was a conflict between the exhibit 300 

data and agencies’ answers to our questions regarding plans for 

delivering functionality in fiscal year 2014, we presented the conflict to the 

agencies and obtained clarification. We determined that the data were 

sufficiently reliable for the purpose of this report, which is to determine the 

extent to which the selected investments planned to deliver functionality 

every 6 and 12 months, during fiscal years 2013 and 2014.

14

 11OMB, Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2013); Guidance on Exhibits 53 and 300—Information Technology and E-Government

(2012); and Guidance on Exhibit 300—Planning, Budgeting, Acquisition, and Management 

of Information Technology Capital Assets (2011).

as well as 

practices for and challenges in effectively applying Agile development 

12VA, Project Management Accountability System Guide 4.0 (Nov. 7, 2012).

13SEI, CMMI-ACQ, Version 1.3 (November 2010).

14GAO, Information Technology: Critical Factors Underlying Successful Major 

Acquisitions, GAO-12-7 (Washington, D.C.: Oct. 21, 2011).
Appendix I: Objectives, Scope, and 

Methodology

Page 44 GAO-14-361 Incremental Development 

methods.15 Additionally, we compared the factors to OMB’s 25 Point 

Implementation Plan to Reform Federal Information Technology 

Management.16

Further, because Defense guidance encourages investments to deliver 

functionality every 12-18 months where possible,17

We conducted this performance audit from May 2013 to May 2014, in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

we asked the selected 

investments and officials from Defense’s Office of the CIO to identify the 

key factors that have both enabled and inhibited their efforts to deliver 

functionality for their major IT investments every 12-18 months. We 

compared the information we received to the eight factors the five 

selected agencies commonly identified as enabling incremental 

development during a 6-month period and the seven factors commonly 

identified by the five agencies as inhibiting incremental development 

during a 6-month period. We also performed a content analysis of the 

information we received in order to identify additional factors.

 15GAO, Software Development: Effective Practices and Federal Challenges in Applying 

Agile Methods, GAO-12-681 (Washington, D.C.: July 27, 2012).

16OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010).

17See, for example, Defense, Office of the Secretary of Defense, Guidance for Fiscal Year 

2013 IT Budget Submissions (Aug. 9. 2011).
Appendix II: Selected Investments’ Plans for 

Delivering Functionality

Page 45 GAO-14-361 Incremental Development 

Below is the list of investments that are included in this review, as well as 

whether each planned to deliver functionality every 6 and 12 months for 

fiscal years 2013 and 2014.

Table 7: Selected Investments and Associated Plans for Delivering Functionality Every 6 and 12 Months, for Fiscal Years 2013 

and 2014

Agency Investment name

Plans to deliver 

functionality every 6 

months in fiscal years 

2013 and 2014

Plans to deliver 

functionality every 12 

months in fiscal years 

2013 and 2014

Defense Advanced Field Artillery Tactical Data System X

Air Force Nuclear Command, Control and 

Communications Minimum Essential Emergency 

Communications Network Modernization

Air Force Integrated Personnel and Pay System

Area Common User System Modernization

Aviation Tactical Communication Systems X

Base Information Transport Infrastructure Wired

Battle Control System Fixed

Common Aviation Command and Control System

Consolidated Afloat Networks Enterprise Service

Defense Agencies Initiative

Defense Enterprise Accounting and Management 

System

X

Distributed Common Ground System Army

Energy Convergence

Global Combat Support System Army X

Global Command And Control System Army

High Performance Computing Modernization Program

Installation Information Infrastructure Modernization 

Program

Integrated Electronic Health Record X

Joint Battle Command-Platform

Joint Personnel Identification Version 2

Joint Precision Approach and Landing System

Joint Tactical Radio System Airborne and Maritime/Fixed 

Station

Joint Tactical Radio System Handheld, Manpack, and 

Small Form Fit Radios

Joint Tactical Radio System Network Enterprise Domain X

Joint Space Operations Center Mission System

Appendix II: Selected Investments’ Plans for 

Delivering Functionality
Appendix II: Selected Investments’ Plans for 

Delivering Functionality

Page 46 GAO-14-361 Incremental Development 

Agency Investment name

Plans to deliver 

functionality every 6 

months in fiscal years 

2013 and 2014

Plans to deliver 

functionality every 12 

months in fiscal years 

2013 and 2014

Defense Launch Test Range System

Maneuver Control System, V6.4 X

Mid-tier Networking Vehicular Radio

Mission Planning System Increment IV X

Multifunctional Information Distribution System X X

Navstar Global Positioning System

Tactical Data Link System

Teleport Generation 3

Theater Battle Management Core System X

Voice Switching System

Warfighter Information Network Tactical Increment 2 X

Warfighter Information Network Tactical Increment 3

Health and 

Human 

Services

Centers for Disease Control and Prevention Public 

Health Information Network: BioSense

Centers for Medicare and Medicaid Services Business 

Rules Enterprise Service

X X

Centers for Medicare and Medicaid Services Center for 

Consumer Information and Insurance Oversight 

Enterprise IT Data Management Investment

X X

Centers for Medicare and Medicaid Services Center for 

Consumer Information and Insurance Oversight 

Healthcare Insurance Exchange IT Investment

X X

Centers for Medicare and Medicaid Services Healthcare 

Quality End Stage Renal Disease Systems

X X

Centers for Medicare and Medicaid Services 

International Classification of Diseases, 10th Revision 

Initiative

Centers for Medicare and Medicaid Services Innovation 

Core Systems

Centers for Medicare and Medicaid Services Medicaid 

and Children’s Health Insurance Program Business 

Information and Solutions

X

Centers for Medicare and Medicaid Services Master Data 

Management

X

U.S. Food and Drug Administration Center for Drug 

Evaluation and Research MedWatch Plus

X X

U.S. Food and Drug Administration’s Office of Regulatory 

Affairs Automated Laboratory Management

X X
Appendix II: Selected Investments’ Plans for 

Delivering Functionality

Page 47 GAO-14-361 Incremental Development 

Agency Investment name

Plans to deliver 

functionality every 6 

months in fiscal years 

2013 and 2014

Plans to deliver 

functionality every 12 

months in fiscal years 

2013 and 2014

Health and 

Human 

Services

U.S. Food and Drug Administration’s Office of Regulatory 

Affairs Mission Accomplishments and Regulatory 

Compliance Services

X X

Federal Health Architecture X X

Heath Resources and Services Administration Bureau of 

Clinician Recruitment and Service Management 

Information System Solution

X X

Homeland 

Security

U.S. Customs and Border Protection Integrated Fixed 

Towers

U.S. Customs and Border Protection Tactical 

Communications Modernization

Mount Weather Information Technology Services

Federal Emergency Management Agency Disaster 

Assistance Improvement Plan

X

Federal Emergency Management Agency Integrated 

Public Alert and Warning System

X X

Federal Emergency Management Agency National Flood 

Insurance Program Information Technology Phoenix

Immigration and Customs Enforcement TECS 

Modernization

National Protection and Programs Directorate 

Continuous Diagnostics and Mitigation

National Protection and Programs Directorate Next 

Generation Networks Priority Services

X

Transportation Security Administration Technology 

Infrastructure Modernization Program

X

U.S. Coast Guard Logistics Information Management 

System

X

U.S. Citizenship and Immigration Services 

Transformation

X X

Transportation Delphi Version Two

Federal Aviation Administration (FAA) Terminal 

Automation Modernization and Replacement Program 

X X

FAA Wide Area Augmentation System X

FAA En Route Automation Modernization D Position 

Upgrade and System Enhancements

FAA Aviation Safety Knowledge Management

FAA System Approach for Safety Oversight

FAA Traffic Flow Management X X

FAA System Wide Information Management X
Appendix II: Selected Investments’ Plans for 

Delivering Functionality

Page 48 GAO-14-361 Incremental Development 

Agency Investment name

Plans to deliver 

functionality every 6 

months in fiscal years 

2013 and 2014

Plans to deliver 

functionality every 12 

months in fiscal years 

2013 and 2014

Transportation FAA Automatic Dependent Surveillance Broadcast X X

FAA Runway Status Lights X X

FAA Data Communications Next Generation Air 

Transportation System Support

FAA Next Generation Air Transportation System National 

Airspace System Voice System

FAA Common Support Services Weather

FAA Next Generation Air Transportation System 

Air/Ground Communications Segment 2

X X

FAA Next Generation Air Transportation System 

Research and Development Trajectory Based Operations

FAA Next Generation Air Transportation System 

Research and Development Arrivals/Departures at High 

Density Airport

FAA Next Generation Air Transportation System 

Research and Development Flexibility in Terminal 

Environment

FAA Next Generation Air Transportation System 

Research and Development Collaborative Air Traffic 

Management

FAA Next Generation Air Transportation System 

Research and Development Reduce Weather Impact

FAA Next Generation Air Transportation System 

Research and Development Demonstrations and 

Infrastructure

Veterans 

Affairs

Benefits Legacy Veterans Services Network X X

InterAgency 21st Century Core X X

InterAgency 21st Century-One Vet X X

Medical 21st Century My HealtheVet X X

Medical 21st Century TeleHealth X X

Medical Legacy X X

Source: GAO analysis of agency data.
Appendix III: Comments from the Department 

of Defense

Page 49 GAO-14-361 Incremental Development 

Appendix III: Comments from the 

Department of Defense
Appendix III: Comments from the Department 

of Defense

Page 50 GAO-14-361 Incremental Development 
Appendix IV: Comments from the Department 

of Homeland Security

Page 51 GAO-14-361 Incremental Development 

Appendix IV: Comments from the 

Department of Homeland Security
Appendix IV: Comments from the Department 

of Homeland Security

Page 52 GAO-14-361 Incremental Development 
Appendix V: Comments from the Department 

of Veterans Affairs

Page 53 GAO-14-361 Incremental Development 

Appendix V: Comments from the Department 

of Veterans Affairs
Appendix V: Comments from the Department 

of Veterans Affairs

Page 54 GAO-14-361 Incremental Development 
Appendix VI: GAO Contact and Staff 

Acknowledgments

Page 55 GAO-14-361 Incremental Development 

David A. Powner, (202) 512-9286 or pownerd@gao.gov

In addition to the contact name above, individuals making contributions to 

this report included Dave Hinchman (Assistant Director), Deborah A. 

Davis (Assistant Director), Rebecca Eyler, Kaelin Kuhn, Jamelyn Payan, 

Meredith Raymond, Kevin Smith, Andrew Stavisky, and Kevin Walsh.

Appendix VI: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(311289)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.