IRS MANAGEMENT

Cost Estimate for New 

Information Reporting 

System Needs to be 

Made More Reliable

Report to Congressional Committees

January 2012

GAO-12-59

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-12-59, a report to 

congressional committees

January 2012

IRS MANAGEMENT

Cost Estimate for New Information Reporting 

System Needs to be Made More Reliable 

Why GAO Did This Study

The Internal Revenue Service

(IRS) began developing the 

Information Reporting and Document 

Matching (IRDM) program in fiscal year 

2009 to enhance IRS’s ability to 

automatically compare different 

sources of tax information and thus 

improve its capacity to identify and 

address taxpayer noncompliance.

GAO’s May 2011 report recommended 

that IRS follow best practices from the 

GAO’s Cost Estimating and 

Assessment Guide if IRS updated the 

cost estimate for building IRDM 

systems. IRS provided a new cost 

estimate for IRDM in August 2011. In

this report, GAO assessed the extent 

to which (1) the IRDM funding request 

is supported by a reliable cost estimate

and, if not reliably supported, why not; 

and (2) IRS’s practices for capturing

data on IRDM’s actual costs and 

comparing them to estimated costs—

known as earned value management 

(EVM)—generate reliable performance 

data. GAO compared IRS’s 2011 

IRDM cost estimate to criteria in GAO’s 

cost guide and analyzed IRDM’s 

earned value management data. 

What GAO Recommends

GAO recommends that IRS ensure 

that IRDM has a reliable cost estimate, 

require certain project teams to work 

with its Estimation Program Office, 

improve cost estimation guidance, and

improve the reliability of IRDM’s EVM 

data. IRS agreed with one, partially 

agreed with one, and disagreed with 

two of GAO’s recommendations. GAO 

generally disagrees with IRS’s 

concerns, and still believes the

recommendations have merit.

What GAO Found

The 2011 Information Reporting and Document Matching (IRDM) cost estimate, 

used to justify the program’s projected budgets of $115 million for fiscal years

2012 through 2016, generally does not meet best practices for reliability. As 

shown in the table below, the cost estimate did not fully meet any of the four best 

practices for a reliable cost estimate.

Best Practices for a Reliable Cost Estimate and IRDM Assessment

Best practices IRDM rating

Comprehensive: the estimate should cover the 

entire program over its full life-cycle. Partially meets.

Well documented: the estimate should be 

supported by detailed documentation. Minimally meets.

Accurate: the estimate should provide unbiased 

results that are not overly conservative or 

optimistic. Minimally meets. 

Credible: the estimate should check for and 

discuss any limitations. Does not meet.

Source: GAO analysis of IRS’s 2011 IRDM cost estimate. 

For example, the cost estimate minimally meets best practices for a well 

documented estimate because the Internal Revenue Service (IRS) did not

provide detailed support for staff resources, and the cost estimate documentation 

only justified about 6 out of the 86 requested full time equivalent staff for IRDM, 

among other things. If documentation does not provide source data or cannot 

explain the calculations underlying the cost elements, the estimate’s credibility 

may suffer. Although IRS has an independent office of cost estimators that can 

develop and update cost estimates using cost modeling software that generally 

follows GAO’s best practices, this office did not develop the 2011 IRDM cost 

estimate. IRS policy does not require project teams to work with the office to 

update cost estimates. Additionally, IRS’s cost estimation guidance for project 

managers is inconsistent regarding how cost estimates should be related to a 

budget, an inconsistency that could lead to budget requests that do not 

accurately estimate program funding needs.

The IRDM program’s earned value management (EVM) data did not meet data 

reliability criteria in the areas GAO reviewed. For example, the IRDM project 

schedule was not properly sequenced—meaning activities were not properly 

linked in the order in which they are to be carried out. In addition, surveillance

was not conducted on IRDM’s EVM system, as required by the Office of 

Management and Budget and the Department of the Treasury. Surveillance 

involves having qualified staff review an EVM system. Because IRDM’s 2011 

cost estimate is based on unreliable EVM data, it does not provide adequate 

support for IRDM’s budget requests. Until IRS addresses deficiencies in the EVM 

data, it cannot provide a reliable cost estimate for IRDM. 

View GAO-12-59. For more information, 

contact Michael Brostek at (202) 512-9110 or 

brostekm@gao.gov. 
Page i GAO-12-59 IRS Management

Letter 1 

Background 3

IRDM’s 2011 Cost Estimate Does Not Meet Best Practices for 

Reliability and Does Not Fully Support the Program’s Budget 9

Unreliable EVM Data Raise Additional Concerns about IRDM Cost

Estimate 23

Conclusions 27

Recommendations for Executive Action 28

Agency Comments and Our Evaluation 29

Appendix I Scope and Methodology 32

Appendix II Assessment of MITS’s Current IRDM Cost Estimate 35

Appendix III High Level Assessment of the Reliability of IRDM’s EVM System 41

Appendix IV Comments from the Internal Revenue Service 42

Appendix V GAO Contact and Staff Acknowledgments 44

Tables

Table 1: Comparison of IRDM 2007 Preliminary Estimate and the 

2009 SCBE (dollars, in millions) 7

Table 2: IRDM Program Budget and Actual Amount Spent, Fiscal 

Years 2009 through 2011 (dollars, in millions) 8

Table 3: MITS IRDM Cost Estimate Alignment with Best Practices 35

Table 4: IRDM EVM Data Reliability 41

Contents
Page ii GAO-12-59 IRS Management

Figures

Figure 1: 2011 IRDM Cost Estimate Alignment with Best Practices 

for Comprehensiveness 11

Figure 2: 2011 IRDM Cost Estimate Alignment with Best Practices 

for Being Well Documented 13

Figure 3: 2011 IRDM Cost Estimate Alignment with Best Practices 

for Accuracy 15

Figure 4: 2011 IRDM Cost Estimate Alignment with Best Practices 

for Credibility 17

Abbreviations

ANSI American National Standards Institute

DME Development/modernization/enhancement

EPO Estimation Program Office

EVM Earned Value Management

FTE Full-time equivalent

IRDM Information Reporting and Document Matching 

IRS Internal Revenue Service

IT Information technology

MITS Modernization and Information Technology Services

SCBE Solution concept-based estimate

WBS Work Breakdown Structure

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-12-59 IRS Management

United States Government Accountability Office

Washington, DC 20548

January 31, 2012

The Honorable Susan M. Collins

Ranking Member

Committee on Homeland Security and Government Affairs

United States Senate

The Honorable Darrell E. Issa

Chairman

The Honorable Elijah E. Cummings

Ranking Member

Committee on Oversight and Government Reform

United States House of Representatives

The financing of the federal government depends largely on the Internal 

Revenue Service’s (IRS) efforts to collect taxes. To help carry out this 

work, IRS initiated the Information Reporting and Document Matching 

(IRDM) program in fiscal year 2009. IRS plans for the IRDM program to 

build information technology (IT) systems that automatically compare—

that is, match—different sources of tax information to improve tax 

compliance.1 In May 2011, we issued a report that assessed IRS’s 2009 

cost estimate for building IRDM IT systems, and found that it did not fully 

meet best practices.2

 

1

IRDM systems will automatically match income or expense information reported by third 

parties to income or expenses reported on a taxpayer’s tax returns. For example, for 

certain securities sales, brokers must report to the IRS cost basis information (generally, 

the difference between the gross proceeds from the securities sale and the original 

purchase price, net any fees or commissions) on the information return, Form 1099-B, 

“Proceeds from Broker and Barter Exchange Transaction.” IRDM systems will then 

compare this information to an individual’s tax return data on securities sales, such as 

found on the Form 1040 Schedule D, “Capital Gains and Losses,” enabling it to identify 

potential compliance problems. 

We recommended that for any future updates to the 

IRDM cost estimate, IRS ensure that the revised estimate be developed 

in a manner that reflects the four characteristics of a reliable cost estimate 

2

GAO, Information Reporting: IRS Could Improve Cost Basis and Transaction Settlement 

Reporting Implementation, GAO-11-557 (Washington, D.C.: May 19, 2011).

 
 

Page 2 GAO-12-59 IRS Management

described in our report.3

Having a reliable cost estimate—a summation of individual cost elements

using established methods and valid data—is vital for making informed 

budgetary decisions and ensuring that a project is implemented as 

planned. In an environment where federal funds are scarce, it is 

imperative that IT projects, such as IRDM, are implemented as planned, 

not only because of their value to the government,

Since then, IRS provided a new cost estimate for 

IRDM in August 2011.

4

This report builds on our May 2011 report

but also because 

every dollar spent on one program will mean one less to fund other 

efforts. Our objectives were to determine the extent to which: (1) the 

IRDM funding request is supported by a reliable cost estimate and, if not 

reliably supported, why not; and (2) IRS’s practices for capturing IRDM’s 

actual costs and comparing them to estimated costs—known as “earned 

value management” (EVM)—generate reliable performance data. We are 

making four recommendations that warrant management’s consideration.

5 and further analysis conducted 

from June 2011 to August 2011, which was used to provide technical 

assistance to Congress. For this report, we compared the current 2011 

IRDM cost estimate to the best practices in our Cost Estimating and 

Assessment Guide.

6

 

3

A reliable cost estimate is (1) comprehensive, (2) well documented, (3) accurate, and (4) 

credible. See GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 

Developing and Managing Capital Program Costs,

We also compared IRS guidance on cost estimation 

to criteria from our cost guide. We shared our cost guide as well as our 

preliminary analysis of the IRDM 2011 cost estimate with program 

officials. When warranted, we updated our analyses based on the 

agency’s response and additional documentation provided to us. To 

assess IRDM’s practices for capturing actual costs and comparing them 

to estimated costs, we used high level criteria on EVM data reliability from 

our cost guide; EVM is a project management approach that uses actual 

GAO-09-3SP (Washington, D.C.: 

March 2009), for a full description of each of the four characteristics. 

4

The IRDM system will increase information that is available for compliance purposes and 

potentially raise significant revenue. Two components of the IRDM program, cost basis 

and transaction settlement reporting, are expected to generate more than $16 billion over 

the next 10 years, according to the Joint Committee on Taxation in 2008.

5

GAO-11-557. 

6

See GAO, Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: Mar. 9, 2009).
 

Page 3 GAO-12-59 IRS Management

project data to provide reports on project status. We also compared 

IRDM’s EVM system with guidance in the Department of the Treasury’s 

Earned Value Management Guide. As part of that analysis, we assessed 

IRDM’s implementation of three guidelines from the American National 

Standards Institute (ANSI), which Treasury’s guidance states that IRDM 

should be following. We selected the three guidelines to represent some 

of the fundamental steps for maintaining a reliable EVM system, as 

identified in our cost guide, and because these guidelines are also 

Treasury Department requirements. For both research objectives, we 

interviewed officials with IRS’s Modernization and Information Technology 

Services (MITS) division, including those from the IRDM Program 

Management Office, and the Investment Planning and Management 

Office, which includes the Estimation Program Office (EPO).7

We conducted this performance audit from August 2011 through January 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives. We determined that the 

IRS data we used were sufficiently reliable for our purposes. We also 

made appropriate attribution indicating the sources of the data. We are 

making recommendations to IRS to improve data reliability in the future. 

See appendix I for more information on our scope and methodology.

IRS initiated the IRDM program, in part, to implement new information 

reporting requirements, but more generally to increase voluntary 

compliance with tax laws by expanding and maximizing IRS’s ability to 

match existing and future information returns with tax return data and 

establishing a new business information matching program. Previously, 

IRS had only matched information returns to individuals’ and sole 

proprietors’ tax returns.8

 

7

EPO is IRS’s cost estimation organization. 

Under IRDM, IRS plans to build several new IT 

8

A sole proprietor is an individual who owns an unincorporated business by himself or 

herself.

Background 

IRDM Program
 

Page 4 GAO-12-59 IRS Management

systems and enhance some existing systems as well as implement 

numerous organizational and process changes. IRS plans for IRDM to 

use information returns to identify individual and business tax returns that 

are likely sources of revenue, which the current individual tax return 

matching system is not designed to identify. IRDM implementation is led 

by IRS’s Small Business/Self Employed division and MITS, which is 

leading the IRDM IT system development.9

Cost estimates are a vital factor for sound management decision making 

and they aid in the formation of a project’s budget. IRS uses cost 

estimates, in part, to justify budget requests and prioritize the selection of 

IT projects for possible funding. After an IT project is approved, the cost 

estimate is later used as a starting point for developing the performance 

measurement baseline for EVM, a project management approach that, if 

implemented appropriately, provides management important tools such 

as objective reports of project status and early warning signs of 

impending schedule delays and cost overruns. Data from a reliable 

performance management system, such as EVM, are necessary inputs 

for an updated cost estimate, among other things.

OMB issued guidance on managing IT projects, which discusses cost 

estimation and refers to our cost guide for how to meet cost estimating 

requirements.10

 

9

For more information on the IRDM program, including IT system development plans, see 

OMB guidelines state that cost estimates should be 

continuously updated based on the latest information available to ensure 

that they are current, accurate, and valid. According to our cost guide, 

effective program and cost control requires ongoing revisions to the cost 

estimate, budget, and projected estimates at completion. Specifically, our 

guide states that estimates should be continuously updated with actual 

costs incurred to that point so that significant cost, schedule, or 

performance variances can be examined. In addition, it says that cost 

estimates should be updated to reflect significant changes to a project’s 

GAO-11-557. 

10Office of Management and Budget, Capital Programming Guide: Supplement to Circular 

A-11, Planning, Budgeting, and Acquisition of Capital Assets, (Executive Office of the 

President, Washington, D.C.: August 2011).

IRS Cost Estimation 

Process and Guidance
 

Page 5 GAO-12-59 IRS Management

scope or specifications and when certain projects approach key 

milestones.11

Within MITS, project managers and EPO are involved in estimating 

program costs. EPO is an independent group of cost estimation experts 

that assists project teams by developing and updating cost estimates 

using a standard documented process. Project managers are responsible 

for maintaining a program’s cost estimate. EPO only becomes involved in 

updating a program’s cost estimate at the request of project managers, 

according to EPO officials.

IRS procedures for developing, using, and updating cost estimates and 

EVM are described in several guidance documents, specifically:

• EPO’s Estimator’s Reference Guide, which is used by EPO staff, is 

the general resource on the processes and procedures for developing 

and delivering IT cost estimates. It discusses the technical aspects of 

updating cost estimates, such as what documents are used in cost 

modeling once a project has begun.

• IRS’s Information Technology Investment Planning and Management 

Guide (Investment Guide) outlines the framework for selecting, 

managing, and evaluating IRS IT projects. The Investment Guide

includes discussions of how IT projects are selected using cost 

information, and how managers should use cost information to 

monitor a project. Project managers are responsible for managing 

cost, schedule, and performance for a project.

• MITS’s Estimation Procedures document describes IRS’s 

organizational approach to cost estimation, applicable to all IRS 

projects. The document is directed at project managers and it includes 

discussions of the steps and staff roles necessary to develop an 

estimate, and the circumstances when EPO typically becomes 

involved with updating a cost estimate. The document states that 

project managers are responsible for monitoring project progress and 

suggests initiating assistance from EPO if a project meets certain 

thresholds.

• The Department of the Treasury’s Earned Value Management Guide

provides guidance for implementing EVM on major IRS projects.

Program or project managers have the ultimate responsibility for 

 

11GAO-09-3SP. 
 

Page 6 GAO-12-59 IRS Management

implementing and monitoring the EVM system for their program or 

project.

IRS developed two cost estimates for IRDM early in the program. In 2007, 

IRS developed a preliminary cost estimate for budgetary purposes when 

very little program information was available (referred to in this report as 

the 2007 preliminary cost estimate).12 As shown in table 1, in 2007, IRDM 

system development was estimated to cost about $5 million in fiscal year 

2009 and about $23 million per year thereafter.13 In 2009, EPO developed 

a solution concept-based estimate (SCBE, referred to in this report as the 

2009 SCBE),14 which was more rigorous than the 2007 preliminary 

estimate. The 2009 SCBE was developed before program implementation 

began, when MITS had more information than it did in 2007, but system 

design plans were still under development.15 The 2009 SCBE was about 

$36 million less through the first 4 years of the project than the 2007 

preliminary estimate.

 

12In our May report, GAO-11-557, based on information from MITS officials, we noted an 

initial estimate developed in 2008 for IRDM’s initial budget request. We obtained 

additional information on IRDM cost estimates indicating that the initial estimate was 

provided for the budget request in 2007, and subsequently confirmed this information with 

MITS officials.

13IRS plans for IRDM system development funding to continue through FY 2016.

14A solution concept based estimate relies on a document, referred to as a solution 

concept, which explains a project proposal’s functional scope and technical solution.

15The 2009 IRDM SCBE did not follow MITS’s standard cost estimation process because 

of time constraints. For this estimate, MITS heavily modified its estimation techniques and 

used broad assumptions.

IRDM Program Cost 

Estimates
 

Page 7 GAO-12-59 IRS Management

Table 1: Comparison of IRDM 2007 Preliminary Estimate and the 2009 SCBE (dollars, in millions)

IRDM Estimate

Fiscal year Total, FY 

2009 

through FY 

2009 2010 2011 2012 2013 2014 2015 2016 2012 

2007 

preliminary 

estimatea $5.1 $23.0 $22.9 $23.2 $23.0 $23.0 $23.0 $23.0 $74.2

2009 SCBEb

$2.6 $13.4 $13.3 $9.3

Not 

estimated

Not 

estimated

Not 

estimated

Not 

estimated $38.6

Difference 

between the 

2007 

preliminary 

estimate and 

SCBE $2.5 $9.6 $9.6 $13.9 N/A N/A N/A N/A $35.6

Source: GAO analysis of IRDM’s Exhibit 300 and the 2009 SCBE.

a

Annual costs based on the preliminary estimate are reported in the Exhibit 300. The Exhibit 300 is a 

document required by OMB to support IT projects. It includes the project’s desired outcome and 

budget justification. The 2007 preliminary estimate provided information from fiscal year 2009 through 

fiscal year 2016.

b

The 2009 SCBE only went through fiscal year 2012.

IRS used the 2007 preliminary estimate to justify the initial IRDM budget.

According to a MITS official, the 2009 SCBE was not used for budgetary 

purposes because program specifications were undergoing modification 

in fiscal year 2010, requiring the full $23-million per-year funding. For 

example, in 2010, IRS made several changes to the complexity of the 

IRDM program, which included dividing it into four projects. One of the 

projects required restructuring as a separate development effort, using 

different resources and technologies. Changes also necessitated using 

new software development methods not previously used within IRS, and 

additional contracting support. According to MITS officials, such changes 

increased IRDM funding needs above the amounts supported by the 

2009 SCBE. As a result, IRS did not revise its initial funding request for 

future fiscal years using the SCBE and instead relied on the 2007 

preliminary estimate.

From fiscal years 2009 through 2011, IRS received about $52 million in 

total funding for IRDM, of which the IRS had spent $46 million (when 

accounting for $2.6 million carried over from fiscal year 2009 funds and 

$5.8 million carried over from fiscal year 2010), through fiscal year 2011, 

as shown in table 2.
 

Page 8 GAO-12-59 IRS Management

Table 2: IRDM Program Budget and Actual Amount Spent, Fiscal Years 2009 

through 2011 (dollars, in millions)

Fiscal year Total FY 2009 

2009 2010 2011 through 2011 

Budget $5.6 $22.9 $23.0 $51.5

Actual amount spent from budget $3.0 $15.8 $18.8 $37.6

Difference between budget and 

amount spenta $2.6 $7.1 $4.2 $13.9

Source: IRS officials, IRDM budget documents, and actual spending data from IRS’s financial information system.

a

The actual amount spent does not include money from the previous fiscal year that was carried over 

and spent. The amounts are not cumulative. In FY 2010, IRDM spent $2.6 million in funds carried 

over from FY 2009, and in FY 2011 IRDM spent $5.8 million of funds carried over from FY 2010.

In our May 2011 report,16

Over the summer of 2011, MITS provided us with additional cost 

information. Officials referred to these documents as IRDM’s new cost 

estimate, and they were used in supporting IRS’s fiscal year 2012 IRDM 

budget. Consequently, in this report, we refer to the materials provided to 

us as the 2011 cost estimate. This estimate was not a SCBE or 

developed by EPO. The 2011 cost estimate was based on several data 

sources, including IRDM’s Exhibit 300, EVM data, spend plans, and 

schedule with work breakdown structures (WBS).

we assessed the 2009 SCBE because it was 

the most rigorous IRDM cost estimate available at the time and the 2007 

preliminary estimate lacked documentation for a complete review. We 

found that the 2009 SCBE did not fully follow best practices. We 

recommended that if IRS updated the cost estimate, it should follow best 

practices from our cost guide. In response to our report, IRS said it would 

update the 2009 SCBE. IRS subsequently decided not to revise the 

estimate because, according to officials, they already have a plan, 

schedule and funding, the program is not over-budget, and the risks 

associated with IRDM, and the program’s size, do not warrant an update. 

17

 

16

MITS projects IRDM 

GAO-11-557. 

17The Exhibit 300 is a document required by OMB to support IT project budgets. It 

includes the project’s desired outcome and budget justification. A WBS shows the 

requirements that must be accomplished to develop a program, and provides the basis for 

identifying resources and tasks for developing a program cost estimate. It provides a basic 

framework for estimating costs, developing schedules, identifying resources, determining 

where risks may occur, and providing the means for measuring program status using 

EVM.
 

Page 9 GAO-12-59 IRS Management

to cost $115 million for fiscal years 2012 through 2016, or about $23 

million per year.

According to best practices established by our cost guide, a cost estimate 

should be comprehensive, well documented, accurate, and credible.18

The estimate partially meets best practices for a comprehensive cost 

estimate, as shown in figure 1.

We 

assessed the 2011 cost estimate against cost estimation best practices 

because IRS told us the estimate was used to support its budget requests 

for fiscal year 2012 and beyond. While the 2011 IRDM cost estimate 

shows some characteristics of a reliable cost estimate, it does not fully 

meet best practices.

19 It reflects the current program schedule 

and contains information about the program’s technical characteristics. 

The estimate provided some details about costs for IRDM’s fiscal year

2012 budget request, but the cost estimate does not cover the program’s 

entire life-cycle.20

 

18

Without fully accounting for life-cycle costs, 

GAO-09-3SP. 

19We rated the extent to which IRS met each best practice on the following scale: “Meets,” 

IRS provided complete evidence that satisfies the entire criterion; “Substantially meets,” 

IRS provided evidence that satisfies a large portion of the criterion; “Partially meets,” IRS 

provided evidence that satisfies about half of the criterion; “Minimally meets,” IRS provided 

evidence that satisfies a small portion of the criterion; and “Does not meet,” IRS provided 

no evidence that satisfies any of the criterion. See app. I for our full scope and 

methodology.

20IRS plans for IRDM development to continue through fiscal year 2016.

IRDM’s 2011 Cost 

Estimate Does Not 

Meet Best Practices 

for Reliability and 

Does Not Fully 

Support the Program’s 

Budget 

IRDM’s Fiscal Year 2012 

and Projected Budget 

Requests Are Not 

Supported by a Reliable 

Cost Estimate
 

Page 10 GAO-12-59 IRS Management

management may have difficulty successfully planning program resource 

requirements and making informed resource-planning decisions. IRS 

defined assumptions used to estimate some IRDM costs, but did not 

provide the assumptions used to estimate labor or program operations 

costs. Furthermore, IRS did not include ground rules21 used to develop 

the estimate. Unless ground rules and assumptions are clearly defined, 

the cost estimate will not have a basis to identify and mitigate areas of 

potential risk.

 

21Ground rules are a set of estimating standards that provide guidance and minimize 

conflicts in definitions, while assumptions are judgments about past, present, or future 

conditions that may affect the estimate. 
Interactive graphic Figure 1: 2011 IRDM Cost Estimate Alignment with Best Practices for Comprehensiveness

Page 11 GAO-12-59 IRS Management

Rollover each below to see further information on cost estimation best practices.

Directions:

Print instructions A full text version of this graphic is available in appendix II.

Source: GAO analysis of IRS’s 2011 IRDM cost estimate and GAO-09-3SP.

A comprehensive cost estimate:

Best practices characteristics

Assessment of whether best 

practices are met Effect

Has a product-oriented work 

breakdown structure (WBS), 

traceable to the program’s 

technical scope, at an 

appropriate level of detail. 

Documents all cost-influencing ground rules and 

assumptions. 

Completely defines the 

program, reflects the 

current schedule, and is 

technically reasonable. 

Includes all life-cycle costs. The estimate includes costs through FY 

2014, but IRS plans for the project to continue through FY 2016, and detailed costs 

are only provided for FY 2012. 

Minimally meets

No ground rules were documented. Several 

documents included assumptions, but none 

discussed associated risks.

Minimally meets

Each of the four IRDM projects has a 

product-oriented WBS, but the WBSs are 

not consistent and they are not traceable to 

the cost estimate. 

Partially meets

The estimate reflects the current project 

schedule and contains high-level information about technical specifications but lacks 

details that would completely define the 

program.

Partially meets

Unless ground rules and assumptions are clearly 

documented, the cost estimate will not have a basis for 

assessing potential risks. Furthermore, the estimate cannot 

be reconstructed when the original estimators are no longer 

available. 

A WBS provides a necessary framework for the program to 

develop a schedule and cost plan that can easily track 

technical accomplishments. A standard, product-oriented 

WBS facilitates the tracking of resource allocations and 

expenditures, which can give the agency insight to reliably 

estimate the cost of future similar programs.

Understanding the program— including the acquisition 

strategy, technical definition, characteristics, system 

design features, and technologies to be included—is key 

to developing a credible cost estimate. Without these data, 

the cost estimator will not be able to identify the technical 

and program parameters that will bind the cost estimate.

A life-cycle cost estimate should encompass all past, 

present, and future costs for every aspect of the program, 

regardless of funding source, including all government and 

contractor costs. Life-cycle cost estimates can enhance 

program managers’ decision making by allowing them to 

evaluate design trade off studies on a total cost basis as 

well as on a technical and performance basis. 

Partially meets best practices for a

comprehensive cost estimate.

Overall assessment:
 

Page 12 GAO-12-59 IRS Management

The estimate minimally meets best practices for a well documented cost 

estimate, as shown in figure 2. IRS provided supporting information for 

some staff resources, but detailed data for the staffing level requested for 

fiscal year 2012 was missing. The cost estimate documentation says that 

the labor cost justification was captured in the resource loaded project 

schedules,22 but we found that these schedules only justified about 6 out 

of the 86 requested full-time equivalent (FTE) staff for IRDM. 

Furthermore, although IRS officials cited the WBS as the basis for cost 

projections, we found no evidence linking the WBS to cost. Multiple 

documents linked software and hardware specifications to cost, but they 

did not provide consistent cost information. As a best practice, 

documentation should describe the source data used, the estimating 

methodology, and show step-by-step how the estimate was developed. 

Without a well documented cost estimate, the program’s credibility may 

suffer because the documentation cannot explain the rationale of the 

methodology or the calculations underlying the cost elements.

 

22A resource loaded schedule is a schedule with resources of staff, facilities, and 

materials needed to complete the activities that use them; it should be based on the 

project’s WBS. 
Interactive graphic Figure 2: 2011 IRDM Cost Estimate Alignment with Best Practices for Being Well Documented

Page 13 GAO-12-59 IRS Management

Rollover each below to see further information on cost estimation best practices.

Directions:

Print instructions A full text version of this graphic is available in appendix II.

Source: GAO analysis of IRS’s 2011 IRDM cost estimate and GAO-09-3SP.

A well documented cost estimate should:

Best practices characteristics

Assessment of whether best 

practices are met Effect

Discuss the technical 

baseline description.

Provide evidence of 

management review and 

acceptance.

Describe the calculations 

and the methodology used 

to derive each element’s 

cost. 

Capture the source data 

used, the reliability of the 

data, and how the data were 

made compatible with other 

data in the estimate.

Data sources are listed, but the estimate is 

not consistent with the source data that it 

cites bringing into question the reliability of 

the data.

Minimally meets

Documentation links hardware and software 

specifications to costs, but project-specific 

costs are only provided for FY 2012. 

Minimally meets

Labor cost calculations are described at a 

high level, but the staffing level is not traceable to the schedule. Hardware and software 

calculations are not described, and costs are 

not consistent across documents.

Minimally meets

Documentation does not fully explain how 

IRS derived estimated costs. 

Partially meets

Because the technical baseline is intended to serve as the 

basis for developing a cost estimate, it should be discussed 

in the cost estimate documentation. Without a technical 

baseline, the cost estimate will not be based on a comprehensive 

program description and will lack specific information regarding 

technical and program risks.

Without good documentation, management and oversight 

organizations will not be convinced that the estimate is 

credible; supporting data, lessons learned, and reasons 

why costs changed will not be available for future use; 

questions about the approach or data used to create 

the estimate cannot be answered; and the scope of the 

analysis cannot be thoroughly defined. 

Poorly documented cost estimates can cause a program’s credibility to suffer because the documentation 

cannot explain the rationale of the methodology or the 

calculations. Estimates that lack sufficient documentation 

are not useful for updates or information sharing and can 

hinder understanding and proper use.

Data are the foundation of every cost estimate. Depending on data quality, an estimate can range anywhere 

from a mere guess to a highly defensible cost position. 

Data are often in many different forms and need to be 

adjusted before being used. The cost estimator needs 

information about the source and reliability of the data 

in order to know whether the data collected can be used 

directly or need to be modified. 

The estimate was developed by IRDM managers, not cost estimators, and while it was 

reviewed within the IRDM program, there 

is no evidence of review by the top IRDM 

managers. 

Minimally meets

Describe how the estimate 

was developed.

A cost estimate is not considered valid until management 

has approved it. It is imperative that management understand how the estimate was developed, including the risks 

associated with the underlying data and methods. 

Minimally meets best practices for a

well documented cost estimate.

Overall assessment:
 

Page 14 GAO-12-59 IRS Management

The estimate minimally meets best practices for an accurate cost 

estimate, as shown in figure 3. Calculations in the estimate are 

mathematically correct. However, documentation that IRS provided to 

support estimated costs for IRDM hardware and software’s estimated 

costs did not match estimates in IRDM’s spend plans. IRS officials said 

the discrepancies occurred because the spend plans were developed 

using more recent cost information for software purchases that were not 

included in the supporting documentation. Additionally, the estimate does 

not list any confidence levels or provide a range of possible costs. 

According to best practices, unless an estimate is based on an 

assessment of the most likely costs and reflects the degree of uncertainty 

given all of the risks considered, management will not be able to make 

informed decisions. IRDM uses EVM to identify variances between 

planned and actual costs, but as discussed below, we found IRDM’s EVM 

data to be unreliable, and there was no evidence that IRS uses actual 

cost data to evaluate whether cost projections are realistic.
Interactive graphic Figure 3: 2011 IRDM Cost Estimate Alignment with Best Practices for Accuracy

Page 15 GAO-12-59 IRS Management

Rollover each below to see further information on cost estimation best practices.

Directions:

Print instructions A full text version of this graphic is available in appendix II.

Source: GAO analysis of IRS’s 2011 IRDM cost estimate and GAO-09-3SP.

An accurate cost estimate:

Best practices characteristics

Assessment of whether best 

practices are met Effect

Contains few mistakes. 

Is regularly updated to 

reflect significant 

program changes. 

Is properly adjusted for 

inflation. 

Produces unbiased 

results. 

No confidence levels are listed and documents do not provide a range of possible 

costs. Neither risk nor uncertainty is 

mentioned. 

Does not meet

IRS does not maintain an updated IRDM 

cost estimate in a single document. 

However, IRDM costs are tracked through 

the program’s WBS and EVM system, and 

evaluates them to develop spend plans, but 

we found that the EVM data are not reliable. 

Minimally meets

Documentation does not specify 

whether the cost estimate went through 

a quality control process, and there are 

inconsistencies among documents, but 

calculations are accurate.

Partially meets

Documentation contains no evidence that 

the cost estimate is adjusted for inflation. 

Does not meet

If a cost estimate is not updated, it can become more difficult to 

analyze changes in program costs and collect cost and schedule 

data to support future cost estimates. The cost estimate should 

be updated when the technical baseline changes; otherwise, it 

will lack credibility. A properly updated cost estimate can provide 

decision makers with accurate information for assessing alternative decisions. 

Without access to estimate details, one cannot be certain that 

calculations are accurate or expressed consistently. 

Adjusting for inflation is important because in the development 

of an estimate, cost data must be expressed in like terms. If 

a mistake is made or the inflation amount is not correct, cost 

overruns can result. 

A cost estimate is biased if the estimated work is overly conservative or too optimistic. Unless the estimate is based on an 

assessment of the most likely costs and reflects the degree of 

uncertainty given all of the risks considered, management will 

not be able to make informed decisions.

Documents and explains 

variances between 

planned and actual 

costs.

Reflects cost estimating 

experiences from 

comparable programs.

Without a documented comparison between the current estimate 

(updated with actual costs) and the old estimate, cost estimators cannot determine the level of variance between the two 

estimates. That is, the estimators cannot see how well they are 

estimating and how the program is changing over time.

Historical data provides the cost estimator with insight into 

actual costs on similar programs, including any cost growth that 

occurred after the original estimate. As a result, historical data 

can be used to challenge optimistic assumptions and bring more 

realism to a cost estimate.

Variances between planned and actual 

costs, and explanations for the variance 

are documented in EVM data, but the 

EVM data are unreliable. 

Minimally meets

Documentation contains no evidence 

that the estimate is based on a model 

that uses historical records of cost 

estimating and actual experiences from 

comparable programs. 

Does not meet

Minimally meets best practices for an

accurate cost estimate.

Overall assessment:
 

Page 16 GAO-12-59 IRS Management

The estimate does not meet best practices for a credible cost estimate, as 

shown in figure 4. For example, the estimate was not crosschecked23 or 

assessed for risk and uncertainty. According to best practices, an 

estimate without risk and uncertainty analysis24 can be unrealistic 

because it does not assess how the cost estimate would be affected if, for 

example, the schedule slipped, the mission changed, or a proposed 

solution did not meet users’ needs. In addition, IRS did not perform a 

sensitivity analysis.25 Further, there is no evidence that another office 

performed a separate cost estimate—referred to as an independent cost 

estimate—to validate the 2011 cost estimate. In previous work we found 

that because of limited resources, IRS generally only does an additional 

independent cost estimate for its largest programs, and according to 

officials, IRDM is not considered a large enough program in terms of its 

funding level.26 While some of MITS’s cost estimates are done by EPO—

which is independent of program management offices—the 2011 cost 

estimate was done by the IRDM program office.

 

23Cross-checks use alternative estimation methods to see if they produce similar results, 

and could enhance the estimate’s reliability.

24Risk and uncertainty refer to the fact that because a cost estimate is a forecast, there is 

always a chance that the actual cost will differ from the estimate.

25A sensitivity analysis examines the effects on changing assumptions and estimating 

procedures to highlight elements that are cost-sensitive.

26Although IRDM is not among IRS’s largest programs by funding level, it is one of IRS’s 

top seven investments, as determined by funding level and mission impact. According to 

our cost guide, for an estimate to be credible, it must be compared to an independent cost 

estimate. However, program managers and the cost estimating team should define the 

scope of the estimate based on its intended purpose, including the appropriate level of 

detail for an independent cost estimate.
Interactive graphic Figure 4: 2011 IRDM Cost Estimate Alignment with Best Practices for Credibility

Page 17 GAO-12-59 IRS Management

Rollover each below to see further information on cost estimation best practices.

Directions:

Print instructions A full text version of this graphic is available in appendix II.

Source: GAO analysis of IRS’s 2011 IRDM cost estimate and GAO-09-3SP.

A credible cost estimate includes:

Best practices characteristics

Assessment of whether best 

practices are met Effect

Cross-checking of major 

cost elements. 

A comparison to an 

independent cost estimate 

conducted by another 

organization. 

A risk and uncertainty 

analysis.

A sensitivity analysis that 

identifies a range of possible 

costs based on varying 

inputs.

Documentation contains no evidence that a 

sensitivity analysis was performed. 

Does not meet

Documentation contains no evidence that an 

independent cost estimate was performed.

Does not meet

Documentation contains no evidence that 

cross-checks were performed.

Does not meet

Documentation contains no evidence that a 

risk analysis or an uncertainty analysis were 

done.

Does not meet

An independent cost estimate is considered one of the best 

and most reliable methods for validating an estimate. It 

provides an independent view of expected program costs 

that tests the program office’s estimate for reasonableness. 

Without an independent cost estimate, decision makers will 

lack insight into a program’s potential costs because independent cost estimates frequently use different methods 

and are less burdened with organizational bias.

If a cross-check demonstrates that alternative methods 

can produce similar results, then confidence in the estimate increases, leading to greater credibility.

For management to make good decisions, the program 

estimate must reflect the degree of uncertainty, so that a 

level of confidence can be given about the estimate. An 

estimate without risk and uncertainty analysis is unrealistic because it does not assess the variability in the 

cost estimate from such effects as schedules slipping, 

missions changing, and proposed solutions not meeting 

users’ needs. 

Because uncertainty cannot be avoided, it is necessary 

to identify the cost elements that represent the most risk. 

A sensitivity analysis reveals how the cost estimate is 

affected by a change in a single assumption, which helps 

the cost estimator to understand which variable(s) most 

affects the cost estimate. Any sources of variation should 

be well documented and traceable. 

Does not meet best practices for a

credible cost estimate.

Overall assessment:
 

Page 18 GAO-12-59 IRS Management

Because the 2011 cost estimate does not meet best practices, it does not 

provide reliable support for IRDM’s fiscal year 2012 budget request, or 

any of the projected budget requests. IRS officials said current IRS policy 

does not require projects to routinely re-estimate project cost. The IRDM 

program office—which does not use the same software or modeling 

techniques as EPO—relied on spend plans, EVM, and other documents 

to estimate costs. In July 2011, MITS officials said that it would take 90 

days for IRDM and EPO staff to complete a new cost estimate for IRDM. 

When considering FTEs and time, a new cost estimate developed by 

IRDM and EPO staff would require a total of about eight staff months.27

EPO has specialized cost estimation tools, such as software that 

incorporates many best practices from our cost guide, and expertise that 

project teams can leverage to update cost estimates. If used correctly, 

EPO estimation procedures could help IRDM management to maintain 

reliable cost information for use in budget requests. EPO officials said 

they did not work with the IRDM team to maintain an updated SCBE 

because the team did not seek their assistance. IRDM was not required 

to do so because MITS guidance, as of September 2011, does not 

require project teams to consult with EPO when updating a cost estimate.

Without EPO involvement, IRS has less assurance that cost estimate 

updates will follow best practices. Our cost guide states that cost 

estimates should be (1) updated to reflect actual costs and changes (i.e., 

significant modifications to a project’s scope or specifications) in order to 

keep the estimate current as the program passes through new phases 

and milestones and (2) updated if there are significant cost, schedule or 

performance variances. The continual updating of the cost estimate as 

the program matures not only results in a higher-quality estimate, but also 

gives cost estimators the opportunity to collect data for use in future 

estimates as well as incorporate lessons learned.

Our cost guide also states that cost estimation work should be done by a 

central independent estimating organization, and estimators should 

 

27IRS officials said it would take 1.5 FTEs for estimators working for 90 days (about 4.5 

staff months), 1.5 FTEs from IRDM staff working for 60 days to assist with the estimate 

(about 3 staff months), and 0.6 additional IRDM FTEs working for 30 days (about 0.6 staff 

months). According to our calculations using a top government salary rate and general 

benefits rate, the direct staff cost associated with updating the cost estimate would likely 

be less than $200,000.

IRDM Could Benefit From 

EPO Assistance on 

Updating Cost Estimates
 

Page 19 GAO-12-59 IRS Management

monitor programs to determine whether preliminary information and 

assumptions remain relevant and accurate. EPO has the following 

characteristics—unlike the IRDM project team—that could help provide 

more reliable cost estimate updates:

• EPO is able to use robust cost estimation techniques, including the 

SEER-SEM software cost estimation model.28

• EPO, as a whole, has more cost estimation experience than the IRDM 

project team. EPO’s six cost estimators have 43 years of combined 

cost estimation experience. They all have received training in SEERSEM and other cost estimation models. In addition, EPO officials said 

that project teams generally would not have the technical skills to 

update a SCBE using cost estimation models. Although the IRDM 

project team has some cost estimation experience and relevant 

training, IRDM officials do not have SEER-SEM training or 

experience. Further, our analysis of IRDM’s 2011 cost estimate 

illustrates that estimate updates done by project teams may not result 

in reliable cost information. According to EPO officials, an updated 

estimate developed by EPO would also be independent, more holistic, 

and would include elements that project teams may miss.

SEER-SEM analyzes 

project histories and cost relationships to produce cost estimates and 

can estimate costs consistent with best practices—such as adjusting 

for risk and incorporating the results of a sensitivity analysis. When 

EPO estimators validate or update a project’s funding requirements, 

they tailor SEER-SEM to the project and use it to consider actual cost 

data from the project team, according to EPO’s Estimator’s Reference 

Guide. Estimators calibrate the model to include the schedule for 

remaining work and evaluate and revise key cost drivers, according to 

the guidance. As mentioned previously, the 2011 IRDM cost estimate 

did not use a cost estimation model.

• EPO-produced updates can help build a historical record of IRS cost 

estimate data. According to our cost guide, historical data are crucial 

to developing high-quality cost estimates because estimators usually 

develop estimates for new programs by relying on data from programs 

that already exist and adjusting for any differences. EPO officials told 

us that they are working to build a historical database that compares 

estimated costs to actual costs. As IRS’s central estimating 

organization, EPO is uniquely qualified to use cost estimate updates 

 

28SEER mathematical models are commercially available cost estimation models, derived 

from extensive software project histories, behavioral models, and metrics. 
 

Page 20 GAO-12-59 IRS Management

as an opportunity to obtain data that are consistent with other 

estimates and to use the data to build a historical cost database, 

which can ensure that future cost estimates are credible.

According to MITS officials, it is up to project teams to seek EPO 

assistance. However, MITS’s Estimation Procedures document, which 

provides cost estimation guidance to project teams, suggests some cost 

and schedule variance and project size thresholds that, if exceeded, 

should cause project managers to contact EPO for an updated estimate. 

It recommends EPO involvement in updating estimates when:

• cost or schedule variance are 10 percent or greater for major projects;

• cost or schedule variance are 25 percent or greater for non-major 

projects;29

• a project with development/modernization/enhancement (DME)

or

30

costs greater than $5 million reaches milestone 3.31

IRDM meets the first threshold. Specifically, IRDM meets the IRS’s 

criteria for a “major” project because IRDM’s projected life-cycle costs are 

about $166 million, based on funding projections submitted to OMB. Also, 

according to IRDM EVM data, as of September 30, 2011, the program 

had a greater than 18 percent cost variance and an almost 13 percent 

schedule variance. IRDM officials said they did not work with EPO to 

update the 2009 SCBE because it was not required by current MITS 

guidance.

MITS officials said is it not feasible to require EPO to update all cost 

estimates for IT projects. EPO is a relatively new office that began 

 

29IRS defines major investments as those that, among other things, have an overall lifecycle cost of greater than $50 million or an annual budget of greater than $5 million. 

Investments that do not meet these criteria are considered non-major. 

30DME is a term used by OMB to describe the program cost for new investments, changes 

or modifications to existing systems to improve capability or performance, changes 

mandated by the Congress or agency leadership, personnel costs for investment 

management, and direct support. For major IT investments, this amount should equal the 

sum of amounts reported for planning and acquisition plus the associated FTE costs 

reported in the Exhibit 300.

31A milestone is a point in time when management reviews updated cost, progress, and 

risk information. According to a MITS document, IRDM and eight other programs have 

DME costs exceeding $5 million in calendar year 2011. IRDM’s DME costs exceed $5 

million, but the program is not scheduled to exit milestone 3 until the summer of 2012.
 

Page 21 GAO-12-59 IRS Management

developing cost estimates in 2006. Most of its work has focused on 

developing estimates for proposed projects, rather than updating 

estimates for existing projects. The Estimation Procedures document was 

developed in September 2011, and as of October 2011, officials said 

EPO has six cost estimators on staff. As a result, EPO officials said that 

project teams generally update IT project cost estimates without EPO 

assistance and that, as of October 2011, EPO has been involved in few 

cost estimate updates. However, senior MITS and EPO officials said they 

would like for EPO to have a greater role in cost estimate updates. If IRS 

does not have reliable cost estimate updates, projects may face risks and 

their budget requests may not be adequately justified to inform decision 

making; these outcomes could be even more significant for projects with 

cost or schedule variances or high DME costs, such as IRDM.

MITS guidance documents used by project managers do not clearly 

discuss the appropriate uses of different types of cost estimates. 

According to current guidance used by EPO estimators, non-SCBE cost 

estimates are less rigorous and are not for use in budgets, but as stated 

above, neither the IRDM initial budget request nor the current and 

projected budgets were developed using information from the 2009 

SCBE.

Three IRS guidance documents describe the relationship between cost 

estimates and budgets. However, the documents are directed at different 

audiences, do not present consistent information, and contain different 

levels of detail. Specifically:

• EPO’s Estimator’s Reference Guide, used by EPO estimators, states 

that budgets for IT projects should be established using SCBEs. IRS’s 

SCBEs rely on cost estimation methods that incorporate best 

practices from our cost guide, including considerations of risk, and 

provide a level of confidence associated with the estimate. According

to our cost guide, for management to make good decisions, the 

program estimate must reflect the degree of uncertainty, so that a 

level of confidence can be given about the estimate. The Estimator’s 

Reference Guide also discusses techniques that estimators can use 

to update SCBEs, and aligns that process with annual budget 

submissions. Although this guidance contains many best practices, it 

is directed at cost estimators; therefore, project managers do not 

typically have access to it or use it.

• The MITS Investment Guide, directed at MITS project managers, 

discusses the role of EPO in developing initial rough estimates and 

MITS Guidance Has 

Inconsistencies and Could 

Better Link Cost 

Estimation Procedures 

with Budget Requests
 

Page 22 GAO-12-59 IRS Management

budget-ready SCBEs and states that, if a project does not yet have an 

SCBE, a rough estimate may be used as a placeholder in a budget 

request. The guide requires that MITS staff should work to ensure that 

if an SCBE exceeds an initial rough estimate, the project’s scope and 

SCBE fit within the appropriated budget. However, the guide does not 

discuss how, if at all, a budget request should be adjusted if an SCBE 

provides an estimated cost that is lower than the budget, or how any 

future cost information should be incorporated into budgets.

• The Estimation Procedures document, directed at MITS project 

managers, does not define types of cost estimates or discuss whether 

they are appropriate for budget decisions. EPO officials said they did 

not believe it is necessary to characterize the different types of cost 

estimates in the Estimation Procedures document because they are 

not necessary for defining the organizational approach to estimation, 

which is the intent of the document. The document states that if 

updated cost estimates indicate that a project’s budget needs to 

change, the changes must be approved. However, it does not specify 

who should approve the estimates.

Without consistent guidance about what types of cost estimates are 

appropriate for budget requests, project teams may not use the best 

information available. Our cost guide states that, as a best practice, an 

estimate intended to support budgetary decisions should cover the 

project’s entire life-cycle and should be supported by a description of the 

program’s technical characteristics, which would be found in an SCBE. 

Using a cost estimate that lacks sufficient rigor—such as a preliminary 

cost estimate, instead of an SCBE—could lead to budget requests that do 

not accurately reflect program funding needs. For example, the 2007 

preliminary IRDM estimate lacks an uncertainty analysis, which would 

provide a basis for adjusting the estimate to reflect unknown facts and 

circumstances that could affect costs, and as a result, IRDM managers do 

not have assurance that the program’s funding level remains appropriate.

Further, not providing project managers with guidance on how to 

incorporate new cost information—either from an SCBE that has replaced 

a preliminary estimate or from an updated cost estimate—into budget 

requests could result in requests that do not reflect current or accurate 

funding needs for a project.
 

Page 23 GAO-12-59 IRS Management

IRS provided EVM data in the 2011 IRDM cost estimate to justify its 

budget requests, but we found that the program’s EVM data are not 

reliable in any of the areas we reviewed. Reliable data on actual 

performance, obtained from an EVM system, are a necessary input if an 

updated cost estimate is to be considered accurate and credible. 

Because IRDM’s 2011 cost estimate is based on unreliable EVM data, it 

does not provide adequate support for IRDM’s budget requests. Until IRS 

addresses deficiencies in its EVM data, it cannot provide reliable cost 

estimate updates for IRDM. EVM data reliability deficiencies, such as 

those we observed for IRDM, are common in federal agencies, and we 

have also previously reported on inconsistencies in implementation of 

EVM for IT projects at Treasury bureaus.32

Our cost guide identifies top-level EVM data reliability tasks for IT 

projects, which are also included in OMB guidance.33

• Maintain an EVM system that is compliant with the agency’s scaling of 

American National Standards Institute (ANSI) guidelines. ANSI has a 

national EVM standard, comprised of 32 guidelines, which define 

acceptable methods for agencies to evaluate an EVM system to 

determine if cost, schedule, and technical performance data can be 

relied on for program management.

We assessed IRDM 

EVM data and IRS’s processes against three data reliability tasks:

• Conduct an integrated baseline review for the program. An integrated 

baseline review is an evaluation of a program’s baseline plan to 

 

32In a review of 16 federal programs using EVM, we found that many programs did not 

fully implement practices to ensure data reliability. For example, 13 of the programs had 

deficiencies in program schedules, which undermined the quality of EVM data. 

Additionally, some programs did not conduct an integrated baseline review, or conduct 

ongoing EVM surveillance. The inconsistent application of EVM across the programs 

exists in part because of the weaknesses in some of the agencies’ policies, as well as a 

lack of enforcement of the EVM policy. See GAO, Information Technology: Agencies Need 

to Improve the Implementation and Use of Earned Value Techniques to Help Manage 

Major System Acquisitions, GAO-10-2 (Washington, D.C.: Oct. 8, 2009). In a review of the 

use of EVM at 6 selected Treasury projects, we found that none of the projects fully 

implemented practices to ensure that the data from their EVM system was reliable. The 

review included projects at the Financial Management Service, IRS, the Bureau of Public 

Debt, and the Treasury departmental offices. See GAO, Information Technology: Treasury 

Needs to Better Define and Implement Its Earned Value Management Policy, GAO-08-951 

(Washington, D.C.: Sept. 22, 2008).

33See GAO-09-3SP and Office of Management and Budget, Capital Programming Guide: 

Supplement to Circular A-11, Planning, Budgeting, and Acquisition of Capital Assets 

(Executive Office of the President, Washington, D.C.: August 2011).

Unreliable EVM Data 

Raise Additional 

Concerns about IRDM 

Cost Estimate
 

Page 24 GAO-12-59 IRS Management

determine whether all program requirements have been addressed, 

risks have been identified, mitigation plans are in place, and available 

and planned resources are sufficient to complete the work.

• Using qualified staff, conduct surveillance on the EVM system.

Surveillance is reviewing a contractor’s EVM system to observe ANSI 

compliance and how well a contractor is using its EVM system to 

manage cost, schedule, and technical performance.

Treasury’s EVM guidance requires a project of IRDM’s size to follow an 

abbreviated set of 10 ANSI guidelines, and to conduct surveillance on the 

EVM system. Other departments also scale ANSI guidelines according to 

the size of projects, which could result in some agencies not fully 

following certain best practices. Further, projects like IRDM, according to 

Treasury guidance, only need to complete an independent baseline 

validation—which although not defined in the guidance, appears to be a 

less rigorous version of an integrated baseline review. Following ANSI 

guidelines and conducting an integrated baseline review and surveillance 

can help ensure that EVM data can indicate how well a program is 

performing in terms of cost, schedule, and technical matters. This 

performance information is necessary for proactive program management 

and risk mitigation, and to maintain a reliable cost estimate. Where 

applicable, we assessed IRDM’s EVM data against the standards cited in 

the Treasury guidance.

We assessed IRDM on three ANSI guidelines, which are fundamental 

elements for an EVM system and are included in Treasury’s abbreviated 

10 EVM guidelines.34

For each selected guideline we found:

We found that IRDM’s EVM system is not compliant 

with these guidelines. For an overview of our findings on IRDM’s EVM 

data reliability, see table 4 in appendix III.

• WBS: This ANSI guideline states that authorized work elements for 

the program should be defined, which typically includes using a WBS 

 

34We assessed IRDM’s adherence to ANSI guidelines 1, 6, and 8. For DME projects 

costing less than $20 million, such as IRDM, Treasury EVM guidance states that a project 

only needs to follow a set of 10 ANSI guidelines (numbers 1, 2, 3, 6, 7, 8, 16, 22, 27, and 

28). Treasury’s 10 guidelines include 9 of the 10 guidelines that are considered by experts 

to be the most critical to follow. Treasury’s list does not include guideline 9. For a list of the 

32 ANSI guidelines, see GAO-09-3SP pgs. 212-213.

IRDM’s EVM System Is Not 

Compliant with Key 

Required Guidelines
 

Page 25 GAO-12-59 IRS Management

tailored for effective internal control. Further, a project’s schedule,35

cost estimate, and EVM system should be based on the same WBS, 

according to our cost guide. The WBSs used in IRDM’s schedules do 

not match the WBS used for EVM. The WBSs in four of the five IRDM 

schedules reflect detailed project-level tasks, while the WBS used for 

the EVM data is only broken down by contractor and government 

efforts, and does not include any project-level data. The WBS used to 

inform the 2011 IRDM cost estimate was not broken down by 

contractor or government data, and it did not provide costs for detailed 

tasks. Without a WBS, from which to measure progress and to serve 

as a consistent framework for the schedules and EVM, there is no 

basis for reliable EVM data, according to our cost guide. An IRS 

official said that the WBS in the IRDM schedules is not the primary 

source of financial information for IRDM. Instead, officials use IRS’s 

financial tracking system, which is much less detailed than the 

schedules’ WBS, to obtain project level data for EVM. This technique 

for gathering EVM data at a project level is contrary to the purpose of 

EVM, which is to integrate cost, schedule, and technical data from 

detailed work packages36 that can be monitored for variances against 

the original plan. Since a resource-loaded schedule forms the 

foundation for the EVM baseline, both the schedule and the EVM data 

should be based on the same WBS, according to our cost guide.37

• Sequencing: This ANSI guideline states that projects should have a 

schedule that describes the sequence of work—that is, a list of 

activities in the order in which they are to be carried out—and 

identifies significant task interdependencies required to meet project 

Because the financial tracking system only provides project level data, 

and financial information cannot be traced to the WBS elements in the 

schedules, the cost associated with the project tasks is unknown.

 

35A schedule provides a time sequence for the duration of a program’s activities and helps 

clarify both the dates for major milestones and the activities that drive the schedule. A 

program schedule also provides the vehicle for developing a time-phased budget 

baseline.

36Work packages are detailed tasks that are typically 4 to 6 weeks long.

37The WBS should be used as the outline for the schedule because the WBS defines the 

work in lower levels of detail. Thus its framework provides the starting point for defining all 

activities and tasks that will be used to develop the schedule. Furthermore, by breaking 

the work into smaller, more manageable work elements, the WBS can be used to 

integrate the scheduled activities and costs for accomplishing EVM. A WBS, therefore, is 

an essential part of EVM cost, schedule, and technical monitoring because it provides a 

consistent framework from which to measure progress. 
 

Page 26 GAO-12-59 IRS Management

requirements. All of the IRDM schedules had significant problems with 

sequencing. For example, many predecessor and successor tasks 

were not linked to one another, which are necessary for properly 

sequencing work so that the schedule will update in response to 

changes. Because the schedule is missing so many of these “logic 

links,” it will not automatically recalculate forecasted start and finish 

dates of remaining activities. Thus, any activities that are late will not 

automatically recalculate the dates for affected successor activities.

IRS officials said they are aware of some issues with missing links. As 

a result of these missing links in all of the IRDM schedules we 

reviewed, IRDM’s schedules are not reliable. Because the schedules 

form the basis for the performance measurement baseline38

• Time-phased budget baseline: This ANSI guideline states that a 

program should establish and maintain a time-phased budget 

baseline,

used to 

track cost and schedule variances in an EVM system, the data from 

the IRDM EVM system are not reliable.

39

IRS officials said although they do not have a program level schedule for 

IRDM, the individual project schedules are linked through 

interdependencies. However, we could not identify these links in our 

analysis. IRS officials said the resource-loaded schedules do not show all 

project resources because some resources, such as contractor personnel 

being used across projects, are used in more than one IRDM project. 

Without schedules that include the resources needed to complete tasks, 

IRS was unable to prove that it had established and maintained a timeat the control account level, against which performance 

can be measured. Resources must be accounted for in a schedule in 

order to develop this baseline, according to our cost guide. IRS was 

unable to show evidence that it has established and maintained a 

time-phased budget baseline for IRDM. Specifically, the program does 

not have one overall schedule, and none of the IRDM schedules we 

reviewed were completely resource-loaded.

 

38A performance measurement baseline is used in EVM to detect deviations from the plan 

and give insight into problems and potential impacts.

39The time-phased budget baseline, against which performance is measured, is formed 

from the performance measurement baseline, which is essentially the resource 

consumption plan for the program. Deviations from the baseline identify areas where 

management should focus attention. A performance measurement baseline represents 

the cumulative value of a program’s planned work over time. It takes into account the 

program activities that occur in a sequenced order, based on finite resources, with 

budgets representing those resources spread over time. 
 

Page 27 GAO-12-59 IRS Management

phased budget baseline. This baseline is a critical EVM component for 

measuring IRDM’s performance.

According to Treasury EVM guidance, an independent baseline validation 

should be conducted for a DME project like IRDM. As stated above, GAO 

and OMB consider an integrated baseline review to be a key element of 

EVM data reliability, while Treasury guidance allows projects like IRDM to 

complete a less-rigorous independent baseline validation. IRS did not 

conduct an integrated baseline review or an independent baseline 

validation for IRDM. IRS officials told us that many of the activities 

typically done during an integrated baseline validation were performed—

such as developing a WBS and schedules. However, as previously 

discussed we identified problems with the WBSs and schedules and the 

baseline process. Without a comprehensive integrated baseline review or 

independent baseline validation, and resolving any issues, IRS has not 

sufficiently evaluated the validity of IRDM’s baseline. This calls into 

question the reliability of IRDM’s EVM data, and could affect the 

program’s ability to identify and mitigate risks.

OMB and Treasury require surveillance on EVM systems, and, according 

to our cost guide, surveillance should be conducted to check whether the 

EVM system summarizes timely and reliable cost, schedule, and 

performance information, among other things. IRS officials said IRS is not 

performing surveillance on the IRDM EVM system because they did not 

believe it was required and because, according to officials, IRS does not 

have staff with the necessary technical skills to conduct surveillance. IRS 

officials said Treasury is reviewing whether Department-level surveillance 

would be efficient. It is important for the agency to conduct surveillance of 

EVM systems to ensure that contractors are following their own 

processes and satisfying ANSI guidelines.

The 2011 IRDM cost estimate does not fully meet best practices for a 

reliable estimate. It is important for IRDM to have a cost estimate that 

meets best practices to inform budgetary decisions and ensure that the 

program is implemented as planned. This standard is particularly 

important in a budgetary environment with scarce resources. IRDM’s 

2011 cost estimate could be improved by using EPO’s expertise to 

ensure the cost estimate follows best practices from our cost guide.

Additionally, more consistent guidance on using cost estimates to develop 

budget requests could help program managers for IRDM, as well as other 

IRS Did Not Validate the 

Baseline for IRDM’s EVM 

System

IRS Is Not Conducting 

Surveillance on IRDM’s 

EVM System

Conclusions 
 

Page 28 GAO-12-59 IRS Management

programs, make budget decisions that are supported by reliable cost 

information.

IRS could increase the credibility of IRDM’s 2011 cost estimate by 

ensuring that IRDM’s EVM data are reliable. Such reliability could also 

allow for IRS to update projected costs for the remainder of IRDM’s 

implementation. Using a WBS that is developed using best practices from 

our cost guide could provide a baseline from which to measure progress, 

a key component to an EVM system. Similarly, developing a single 

integrated schedule for IRDM, that contains all resources needed to 

implement the program, could provide more meaningful EVM data.

Finally, providing oversight of the EVM system, through an independent 

baseline validation and EVM surveillance, could help identify potential 

program risks and any possible issues with contractor performance.

To improve the quality of cost and budget information for IRS IT projects, 

we recommend that the Commissioner of Internal Revenue take the 

following four actions: 

1. Ensure that the IRDM life-cycle cost estimate is reliable and that 

budget requests are justified by a reliable cost estimate that follows 

best practices.

2. Require project managers to consult with EPO to determine if projects 

could benefit from EPO assistance in updating cost estimates for 

programs that exceed thresholds recommended by MITS’s Estimation 

Procedures document. For those projects where EPO does not 

update the cost estimate, IRS should require that the decision and 

rationale be documented. EPO should use the information from 

updated cost estimates to develop a historical repository of cost 

estimation data.

3. Review all guidance applicable to cost estimates and take steps to 

ensure that they are consistent. As a first step, IRS guidance should 

require the use of current and reliable project cost estimates to inform 

budget requests, in accordance with the Estimator’s Reference Guide. 

4. Improve the reliability of IRDM’s EVM data, specifically:

• address WBS issues by developing an EVM baseline for IRDM 

that reflects the same WBS as the detailed schedule and IRDM 

cost estimate;

• address sequencing issues and enable the development of a timephased budget baseline by creating a single integrated master 

schedule for IRDM that is properly sequenced and resourceRecommendations for 

Executive Action
 

Page 29 GAO-12-59 IRS Management

loaded so that effective and meaningful EVM data can be 

obtained to better manage the program;

• conduct an independent baseline validation for the IRDM EVM 

baseline; and

• conduct independent surveillance of EVM systems to ensure that 

data are reliable.

We provided a draft of this report to the Commissioner of Internal 

Revenue for his review and comment. We received written comments 

from the Deputy Commissioner for Operations Support, which are 

reprinted in appendix IV. We sought clarification on IRS’s written 

response in regards to whether it agreed with two of our 

recommendations, and on a reference to OMB guidance. IRS provided us 

with additional comments, which are summarized below. In addition, the 

agency provided technical comments, which we incorporated into the 

report as appropriate. IRS agreed with one of our four recommendations, 

partially agreed with another, and disagreed with two. 

While IRS’s comment letter did not address the recommendation, the 

Director of Risk Management in MITS’s Strategy and Planning Office told 

us the agency agrees with the recommendation to require certain IT 

project managers to consult with EPO about updating cost estimates, 

documenting decisions not to update cost estimates, and placing data 

from updated cost estimates in a repository. Similarly, IRS’s comment 

letter did not address our recommendation to ensure that its cost 

estimation guidance is consistent. However, IRS officials said they 

partially agree with this recommendation and have taken steps to ensure

that their estimation practices and procedures follow consistent, 

documented guidance. They noted that in all instances, however, IRS IT 

cost estimates will be based on the best information available at the time 

the estimate is requested or required as opposed to our recommendation 

that IRS require the use of current and reliable cost estimates to inform all 

budget requests. We note in our report that using an unreliable cost 

estimate could lead to budget requests that do not accurately reflect 

program funding needs. Once done in conformance with guidance and 

best practices, current and reliable cost estimates can be maintained 

through normal required monitoring and cost tracking procedures, unless 

significant changes in project circumstances warrant updating the cost 

estimate.

Agency Comments 

and Our Evaluation
 

Page 30 GAO-12-59 IRS Management

IRS disagreed with our recommendation to ensure that the IRDM lifecycle cost estimate is reliable and that budget requests are justified by a 

reliable cost estimate that follows best practices. In its comment letter, 

IRS wrote that implementing the recommendation would require it to 

spend resources that do not directly contribute to the successful 

implementation of the IRDM program, and that the IRDM program is 

within its budget and schedule. At the end of fiscal year 2011, IRDM was 

under budget by more than 18 percent and behind schedule by almost 13 

percent, equal to over 4 months behind schedule, according to IRDM’s 

EVM data. Because the IRDM program does not have a reliable cost 

estimate, budget authorities do not have reliable information to determine

an appropriate funding level. Such variances indicate that the current 

funding level may not be appropriate. As we reported, IRS estimated in 

July 2011 that it would take about 90 days, comprising 8 staff months (or 

a direct staff cost likely less than $200,000, according to our calculations 

using a top government salary rate and general benefits rate), to 

complete a new cost estimate for IRDM. While we agree that federal 

resources are tight, we believe that such an investment could produce 

benefits that not only improve the reliability of the IRDM cost estimate, but

also better ensure that IRS requests the correct amount of resources to 

ensure IRDM fully achieves successful implementation. Further, benefits 

of a new estimate would also stretch beyond that program and provide an 

important foundation for improving IRS cost estimates in general. The 

OMB Capital Programming Guide directs agencies to develop sound cost 

estimates based on our cost guide and states that during the budget 

process, the credibility of the costs will be examined, and OMB and the 

Congress will hold agencies accountable for meeting the schedule and 

performance goals within the cost estimates. More reliable cost estimates 

enable Congress and other budget authorities to make more complete 

and informed decisions. 

IRS also disagreed with our recommendation to improve the reliability of 

EVM data, and stated that OMB’s revisions to Circular A-11 remove EVM 

system requirements due to negative cost benefit. We disagree with IRS, 

as Circular A-11, Appendix J and the Capital Programming Guide still 

contain language that directs agencies to use EVM for major projects. 

Former Circular A-11, section 300.7, instructed agencies to use EVM 

system requirements to identify areas where problems are occurring 

when reporting on ongoing investments. While current guidance for 

Exhibit 300 no longer explicitly discusses EVM reporting on ongoing 

investments under section 300.7, other sections of the guidance still 

direct the use of EVM for managing IT capital assets and state that in 

general cost, schedule and performance goals are to be controlled and 
 

Page 31 GAO-12-59 IRS Management

monitored by using an EVM system. Moreover, our report assessed the 

reliability of IRDM’s EVM data because the data were included in IRDM’s 

2011 cost estimate and the data are used to track the program’s 

progress. Regardless of OMB requirements, any data used for cost 

estimation and program management, particularly when it helps to 

support a budget request, should be reliable. 

We will send copies of this report to the Chairmen and Ranking Members 

of Senate and House committees and subcommittees that have 

appropriation, authorization, and oversight responsibilities for IRS. We will 

also send copies to the Commissioner of Internal Revenue, the Secretary 

of the Treasury, the Chairman of the IRS Oversight Board, and the Acting 

Director of the Office of Management and Budget. Copies are also 

available at no charge on the GAO Web site at http://www.gao.gov. 

If you or your staffs have any questions concerning this report, please 

contact me at (202) 512-9110 or brostekm@gao.gov. Contact points for 

our Offices of Congressional Relations and Public Affairs may be found 

on the last page of this report. GAO staff who made major contributions to 

this report are listed in appendix V.

Michael Brostek

Director, Tax Issues

Strategic Issues 
Appendix I: Scope and Methodology

Page 32 GAO-12-59 IRS Management

This report builds on our May 2011 report1

To assess the extent to which the IRDM funding request is supported by 

a reliable cost estimate, and if not reliably supported, why not, we 

compared the Modernization and Information Technology Services 

(MITS) division’s 2011 IRDM cost estimate with the characteristics of a 

high-quality cost estimate, identified in the GAO Cost Estimating and 

Assessment Guide.

on the Internal Revenue 

Service’s (IRS) Information Reporting and Document Matching (IRDM) 

program and analysis performed during a budget justification review of 

the IRDM program, conducted from June 2011 to August 2011, which we 

provided to Congress as technical assistance. In the May 2011 report, we 

assessed IRDM’s 2009 solution-concept based cost estimate. For the 

budget justification work and this report, we assessed IRDM’s 2007 

preliminary cost estimate. Also for this report, we assessed IRDM’s 2011 

cost estimate.

2 The 2011 IRDM cost estimate documentation 

included spend plans, the Exhibit 300, Earned Value Management (EVM) 

data, project schedules, and other documents.3

 

1

GAO, Information Reporting: IRS Could Improve Cost Basis and Transaction Settlement 

Reporting Implementation, 

Our cost guide, which is 

based on extensive research of best practices for estimating program 

schedules and costs, indicates that a high-quality, valid, and reliable cost 

estimate should be well documented, comprehensive, accurate, and 

credible; we analyzed the cost estimating practices used by MITS against 

these characteristics and rated each characteristic as being either: Met, 

Substantially Met, Partially Met, Minimally Met, or Not Met. To do so, we 

scored each of the individual key practices associated with cost and 

scheduling best practices on a scale of 1-5 (Does Not Meet = 1, Minimally 

Meets = 2, Partially Meets = 3, Substantially Meets = 4, and Meets = 5), 

and then averaged the individual practice scores to determine the overall 

rating. We shared our cost guide as well as our preliminary analysis of the 

IRDM 2011 cost estimate with program officials. When warranted, we 

updated our analyses based on the agency’s response and additional 

GAO-11-557 (Washington, D.C.: May 19, 2011).

2

See GAO, Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 

3

The Exhibit 300 is a document required by the Office of Management and Budget to 

support IT projects. It includes the project’s desired outcome and budget justification.

Earned Value Management is a project management tool that integrates the technical 

scope of work with schedule and cost elements for investment planning and control.

Appendix I: Scope and Methodology
Appendix I: Scope and Methodology

Page 33 GAO-12-59 IRS Management

documentation provided to us. Once we determined that the 2011 cost 

estimate did not fully meet best practices, we determined why this 

occurred. To this end, we interviewed officials in IRDM’s Program 

Management Office and MITS’s Estimation Program Office (EPO). We 

also compared IRS guidance that addresses cost estimation—the EPO’s 

Estimator’s Reference Guide, MITS’s Estimation Procedures document, 

and IRS’s Information Technology Investment Planning and Management 

Guide—to criteria in our cost guide.

To assess the extent to which IRS’s practices for capturing IRDM’s actual 

costs and comparing them to estimated costs, or EVM, generate reliable 

performance data, we compared the EVM data for IRDM and IRS’s 

process for maintaining the data to the high-level EVM data reliability 

tasks outlined in our cost guide.4 We assessed the extent to which 

IRDM’s EVM data adhered to three of the American National Standard 

Institute’s (ANSI)5 32 guidelines; we selected the three guidelines to 

represent some of the fundamental steps for maintaining a reliable EVM 

system, as identified in our cost guide, and because these guidelines are 

also included in the Department of the Treasury’s Earned Value 

Management Guide, which applies to IRS (see appendix III for a list of the 

data reliability tasks and ANSI guidelines we reviewed).6 In situations 

where Treasury’s Earned Value Management Guide did not require 

certain OMB or GAO best practices, we assessed IRDM’s EVM practices 

against the Treasury guidance. To do this analysis, we compared the 

work breakdown structures used in IRDM’s EVM system, schedules, and 

cost estimates and identified differences in each. We also assessed each 

of the five IRDM schedules7

 

4

against scheduling best practices for 

GAO-09-3SP, see pg. 98 for a list of EVM data reliability tasks, we assessed whether 

IRDM is implementing the first three.

5

ANSI has a national EVM standard, comprised of 32 guidelines, that define acceptable 

methods for agencies to evaluate an EVM system to determine if cost, schedule, and 

technical performance data can be relied on for program management. We assessed 

whether IRDM was following guideline numbers 1, 6, and 8. 

6

OMB provides guidance for cost estimation and EVM, but some departments and 

agencies may scale ANSI guidelines to fit specific projects. See Office of Management 

and Budget, Capital Programming Guide: Supplement to Circular A-11, Planning, 

Budgeting, and Acquisition of Capital Assets, (Washington, D.C.: Executive Office of the 

President, August 2011).

7

IRS officials provided five schedules for IRDM, one for each of the four IRDM projects, 

and a schedule depicting IRDM’s progress following IRS Enterprise Lifecycle guidance.
Appendix I: Scope and Methodology

Page 34 GAO-12-59 IRS Management

ensuring that the activities are sequenced and related using network 

logic, as identified in our cost guide.

For both objectives, we interviewed IRS officials in the MITS division, 

specifically, officials from the IRDM Program Management Office and the 

Investment Planning and Management Office, which includes EPO. We 

spoke primarily with officials at IRS Headquarters in Washington, D.C. 

and IRS’s division office in New Carrollton, Maryland, where the officials 

responsible for IRDM are located. To assess the reliability of the cost 

estimate data that we used to support findings in this report, we reviewed 

relevant program documentation, such as cost estimation spreadsheets, 

as available, to substantiate evidence obtained from interviews with 

knowledgeable agency officials. We found the data we used to be 

sufficiently reliable for the purposes of our report. As appropriate, we 

attributed the sources of the data. We are making recommendations to 

IRS to improve data reliability in the future.

We conducted this performance audit from August 2011 through January 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate

Page 35 GAO-12-59 IRS Management

The following table outlines our assessment of the extent to which the 

Internal Revenue Service’s (IRS) 2011 Information Reporting and 

Document Matching (IRDM) program cost estimate meets best practices, 

depicted in figures 1-4.

Table 3: MITS IRDM Cost Estimate Alignment with Best Practices

Best practices 

characteristics

Overall 

assessment

 Assessment of whether

 best practices met Effect 

A comprehensive cost 

estimate:

Partially meets best 

practices for a 

comprehensive cost 

estimate.

Includes all life-cycle costs.

A life-cycle cost estimate 

provides a complete and 

structured accounting of all 

resources and associated cost 

elements required to develop, 

produce, deploy, and sustain a 

particular program. It should 

cover the inception of the 

program through its retirement.

The estimate includes costs 

through FY 2014, but IRS 

plans for the project to 

continue through FY 2016, 

and detailed costs are only 

provided for FY 2012. 

(Minimally meets.)

A life cycle cost estimate should 

encompass all past, present, and 

future costs for every aspect of the 

program, regardless of funding 

source, including all government and 

contractor costs. Life-cycle cost 

estimates that include all costs can 

enhance program managers’ decision 

making by allowing them to evaluate 

design trade off studies on a total 

cost basis as well as on a technical 

and performance basis.

Completely defines the 

program, reflects the 

current schedule, and is 

technically reasonable. 

The cost estimate should be 

based on a documented 

technical baseline description, 

which provides a common 

definition of the program, 

including detailed technical, 

program, and schedule 

descriptions of the system.

The estimate reflects the 

current project schedule and 

contains high-level 

information about technical 

specifications but lacks 

details that would completely 

define the program. (Partially 

meets.)

Understanding the program—

including the acquisition strategy, 

technical definition, characteristics, 

system design features, and 

technologies to be included—is key to 

developing a credible cost estimate. 

Without these data, the cost estimator 

will not be able to identify the 

technical and program parameters 

that will bind the cost estimate.

Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate
Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate

Page 36 GAO-12-59 IRS Management

Best practices 

characteristics

Overall 

assessment

 Assessment of whether

 best practices met Effect 

Has a product-oriented 

work breakdown structure 

(WBS), traceable to the 

program’s technical scope, 

at an appropriate level of 

detail. 

A WBS provides a basic 

framework for a variety of 

related activities like estimating 

costs, developing schedules, 

identifying resources and 

potential risks, and providing 

the means for measuring 

program status using EVM. It is 

product-oriented if it allows a 

program to track cost and 

schedule by defined 

deliverables, such as a 

hardware or software 

component. 

Each of the four IRDM 

projects has a productoriented WBS, but the WBSs 

are not consistent and they 

are not traceable to the cost 

estimate. (Partially meets.)

A WBS provides a necessary 

framework for the program to develop 

a schedule and cost plan that can 

easily track technical 

accomplishments. A standard, 

product-oriented WBS facilitates the 

tracking of resource allocations and 

expenditures, which can give the 

agency insight to reliably estimate the 

cost of future similar programs.

Documents all costinfluencing ground rules 

and assumptions.

Cost estimates are typically 

based on limited information 

and therefore need to be bound 

by ground rules and 

assumptions. Ground rules are 

a set of estimating standards 

that provide guidance and 

common definitions, while 

assumptions are judgments 

about past, present, or future 

conditions that may affect the 

estimate. Any risks associated 

with assumptions should be 

identified and traced to specific 

WBS elements.

No ground rules were 

documented. Several 

documents included 

assumptions, but none 

discussed associated risks. 

(Minimally meets.)

Unless ground rules and assumptions 

are clearly documented, the cost 

estimate will not have a basis for 

assessing potential risks. 

Furthermore, the estimate cannot be 

reconstructed when the original 

estimators are no longer available.

A well documented cost 

estimate should:

Minimally meets 

best practices for a 

well documented 

cost estimate.
Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate

Page 37 GAO-12-59 IRS Management

Best practices 

characteristics

Overall 

assessment

 Assessment of whether

 best practices met Effect 

Capture the source data 

used, the reliability of the 

data, and how the data 

were made compatible 

with other data in the 

estimate.

Data should be collected from 

primary sources. The source, 

content, time, and units should 

be adequately documented. 

Further, data should be 

analyzed to determine 

accuracy and reliability, and to 

identify cost drivers.

Data sources are listed, but 

the estimate is not consistent 

with the source data that it 

cites bringing into question 

the reliability of the data. 

(Minimally meets.)

Data are the foundation of every cost 

estimate. Depending on data quality, 

the estimate can range anywhere 

from a mere guess to a highly 

defensible cost position. Data are 

often in many different forms and 

need to be adjusted before being 

used. The cost estimator needs 

information about the source and 

reliability of the data in order to know 

whether the data collected can be 

used directly or need to be modified.

Describe the calculations 

and the methodology used 

to derive each element’s 

cost.

Documentation should describe 

what calculation methods are 

used, as well as how they were 

applied, and explain any 

anomalies.

Documentation does not fully 

explain how IRS derived 

estimated costs. (Partially 

meets.)

Poorly documented cost estimates 

can cause a program’s credibility to 

suffer because the documentation 

cannot explain the rationale of the 

methodology or the calculations. 

Estimates that lack sufficient 

documentation are not useful for 

updates or information sharing and 

can hinder understanding and proper 

use.

Describe how the estimate 

was developed.

The data supporting the 

estimate should be available 

and adequately documented so 

that the estimate can be easily 

updated to reflect actual costs 

or program changes. 

Labor cost calculations are 

described at a high level, but 

the staffing level is not 

traceable to the schedule. 

Hardware and software 

calculations are not 

described, and costs are not 

consistent across 

documents. (Minimally 

meets.)

Without good documentation, 

management and oversight 

organizations will not be convinced 

that the estimate is credible; 

supporting data, lessons learned, and 

reasons why costs changed will not 

be available for future use; questions 

about the approach or data used to 

create the estimate cannot be 

answered; and the scope of the 

analysis cannot be thoroughly 

defined. 

Discuss the technical 

baseline description. 

A technical baseline description 

provides a common definition 

of the program, including 

detailed technical, program, 

and schedule descriptions of 

the system, for a cost estimate 

to be built on. The data in the

technical baseline should be 

consistent with the cost 

estimate.

Documentation links 

hardware and software 

specifications to costs, but 

project-specific costs are 

only provided for FY 2012. 

(Minimally meets.)

Because the technical baseline is 

intended to serve as the basis for 

developing a cost estimate, it should 

be discussed in the cost estimate 

documentation. Without a technical 

baseline, the cost estimate will not be 

based on a comprehensive program 

description and will lack specific 

information regarding technical and 

program risks.
Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate

Page 38 GAO-12-59 IRS Management

Best practices 

characteristics

Overall 

assessment

 Assessment of whether

 best practices met Effect 

Provide evidence of 

management review and 

acceptance.

There should be a briefing to 

management, including a clear 

explanation of how the cost 

estimate was derived. 

Management’s acceptance of 

the cost estimate should be 

documented.

The estimate was developed 

by IRDM managers, not cost 

estimators, and while it was 

reviewed within the IRDM 

program, there is no 

evidence of review by the top 

IRDM managers. (Minimally 

meets.)

A cost estimate is not considered 

valid until management has approved 

it. It is imperative that management 

understand how the estimate was 

developed, including the risks 

associated with the underlying data 

and methods.

An accurate cost estimate: Minimally meets 

best practices for 

an accurate cost 

estimate.

Produces unbiased results.

Cost estimates should have an 

uncertainty analysis, which 

determines where the estimate 

falls against the range of all 

possible costs.

No confidence levels are 

listed and documents do not 

provide a range of possible 

costs. Neither risk nor 

uncertainty is mentioned. 

(Does not meet.)

A cost estimate is biased if the 

estimated work is overly conservative 

or too optimistic. Unless the estimate 

is based on an assessment of the 

most likely costs and reflects the 

degree of uncertainty given all of the 

risks considered, management will 

not be able to make informed 

decisions.

Is properly adjusted for 

inflation.

Cost data should be adjusted 

for inflation to ensure that 

comparisons and projections 

are valid. Data should also be 

normalized to constant year 

dollars to remove the effects of 

inflation. Also, inflation 

assumptions must be well 

documented.

Documentation contains no 

evidence that the cost 

estimate is adjusted for 

inflation. (Does not meet.)

Adjusting for inflation is important 

because in the development of an 

estimate, cost data must be 

expressed in like terms. If a mistake 

is made or the inflation amount is not 

correct, cost overruns can result.

Contains few mistakes.

Results should be checked for 

accuracy, double-counting, and 

omissions. Validating that a 

cost estimate is accurate 

requires thoroughly 

understanding and 

investigating how the cost 

model was constructed.

Documentation does not 

specify whether the cost 

estimate went through a 

quality control process, and 

there are inconsistencies 

among documents, but 

calculations are accurate. 

(Partially meets.)

Without access to estimate details, 

one cannot be certain that 

calculations are accurate or 

expressed consistently.
Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate

Page 39 GAO-12-59 IRS Management

Best practices 

characteristics

Overall 

assessment

 Assessment of whether

 best practices met Effect 

Is regularly updated to 

reflect significant program 

changes.

The cost estimate should be 

updated to reflect significant 

program changes, such as 

changes to schedules or other 

assumptions. Updates should 

also reflect actual costs so that 

the estimate always reflects the 

current program status.

IRS does not maintain an 

updated IRDM cost estimate 

in a single document. 

However, IRDM costs are 

tracked through the 

program’s WBS and EVM 

system, and IRS evaluates 

them to develop spend 

plans, but we found that the 

EVM data are not reliable. 

(Minimally meets.)

If a cost estimate is not updated, it 

can become more difficult to analyze 

changes in program costs and collect 

cost and schedule data to support 

future cost estimates. The cost 

estimate should be updated when the 

technical baseline changes; 

otherwise, it will lack credibility. A 

properly updated cost estimate can 

provide decision makers with 

accurate information for assessing 

alternative decisions.

Documents and explains 

variances between 

planned and actual costs. 

Variances between planned 

and actual costs should be 

documented, explained, and 

reviewed. For any elements 

whose actual costs or 

schedules differ from the 

estimate, the estimate should 

discuss variances and lessons 

learned.

Variances between planned 

and actual costs, and 

explanations for the variance 

are documented in EVM 

data, but the EVM data are 

unreliable (Minimally meets.)

Without a documented comparison 

between the current estimate 

(updated with actual costs) and the 

old estimate, cost estimators cannot 

determine the level of variance 

between the two estimates. That is, 

the estimators cannot see how well 

they are estimating and how the 

program is changing over time.

Reflects cost estimating 

experiences from 

comparable programs.

The estimate should be based 

on historical cost estimation 

data and actual experiences 

from other comparable 

programs. These data should 

be reliable and relevant to the 

new program.

Documentation contains no 

evidence that the estimate is 

based on a model that uses 

historical records of cost 

estimating and actual 

experiences from 

comparable programs. (Does 

not meet.)

Historical data provides the cost 

estimator with insight into actual costs 

on similar programs, including any 

cost growth that occurred after the 

original estimate. As a result, 

historical data can be used to 

challenge optimistic assumptions and 

bring more realism to a cost estimate. 

A credible cost estimate 

includes:

Does not meet best 

practices for a 

credible cost 

estimate.
Appendix II: Assessment of MITS’s Current 

IRDM Cost Estimate

Page 40 GAO-12-59 IRS Management

Best practices 

characteristics

Overall 

assessment

 Assessment of whether

 best practices met Effect 

A sensitivity analysis that 

identifies a range of 

possible costs based on 

varying inputs. 

A sensitivity analysis examines 

how changes to key 

assumptions and inputs affect 

the estimate. The estimate 

should identify key cost drivers, 

examine their parameters and 

assumptions, and re-estimate 

the total cost by varying each 

parameter between its 

minimum and maximum range. 

Documentation contains no 

evidence that a sensitivity 

analysis was performed. 

(Does not meet.)

Because uncertainty cannot be 

avoided, it is necessary to identify the 

cost elements that represent the most 

risk. A sensitivity analysis reveals 

how the cost estimate is affected by a 

change in a single assumption, which 

helps the cost estimator to 

understand which variable(s) most 

affects the cost estimate. Any 

sources of variation should be well 

documented and traceable.

A risk and uncertainty 

analysis. 

A risk and uncertainty analysis 

recognizes the potential for 

error and attempts to quantify it 

by identifying the effects of 

changing key cost drivers.

Documentation contains no 

evidence that a risk analysis 

or an uncertainty analysis 

were done. (Does not meet.)

For management to make good 

decisions, the program estimate must 

reflect the degree of uncertainty, so 

that a level of confidence can be 

given about the estimate. An estimate 

without risk and uncertainty analysis 

is unrealistic because it does not 

assess the variability in the cost 

estimate from such effects as 

schedules slipping, missions 

changing, and proposed solutions not 

meeting users’ needs.

Cross-checking of major 

cost elements. 

A cross-check is done by using 

a different cost estimation 

method to see if it produces 

similar results.

Documentation contains no 

evidence that cross-checks 

were performed. (Does not 

meet.)

If a cross-check demonstrates that 

alternative methods can produce 

similar results, then confidence in the 

estimate increases, leading to greater 

credibility.

A comparison to an 

independent cost estimate 

conducted by another 

organization.

A second, independent, cost 

estimate should be performed 

by an organization outside of 

the program office’s influence. 

It should be based on the same 

technical baseline, ground 

rules, and assumptions, as the 

original estimate.

Documentation contains no 

evidence that an 

independent cost estimate 

was performed. (Does not 

meet.)

An independent cost estimate is 

considered one of the best and most 

reliable methods for validating an 

estimate. It provides an independent 

view of expected program costs that 

tests the program office’s estimate for 

reasonableness. Without an 

independent cost estimate, decision

makers will lack insight into a 

program’s potential costs because 

independent cost estimates 

frequently use different methods and 

are less burdened with organizational 

bias. 

Source: GAO analysis of IRS’s 2011 IRDM cost estimate and GAO 09-3SP.
Appendix III: High Level Assessment of the 

Reliability of IRDM’s EVM System

Page 41 GAO-12-59 IRS Management

Table 4: IRDM EVM Data Reliability

Data reliability task Was task met? Explanation of GAO assessment

Maintain an earned value management (EVM) 

system that is compliant with the agency’s scaling 

of ANSI guidelines.a

 

No IRDM did not meet the three ANSI guidelines we assessed.

• Work Elements: Define the authorized work 

elements for the program, typically done using 

a WBS.

• No • The WBS used in IRDM’s schedules did not match the 

WBS used for EVM.

• Sequencing: Schedule the authorized work in a 

manner that describes the sequence of work 

and identifies significant task 

interdependencies.

• No • The IRDM schedules did not have proper sequencing; 

related project activities were not linked.

• Time-phased budget baseline: b At the control 

account level, establish and maintain a timephased budget baseline, against which 

program performance can be measured.

• No • IRS was unable to show evidence that it established a 

time-phased budget baseline for IRDM. IRDM does not 

have one overall schedule, and none of the four IRDM 

project schedules were completely resource loaded. 

Conduct an integrated baseline review, or 

independent baseline validation, for the program. c No Neither an integrated baseline review nor an independent baseline validation were conducted for IRDM.

Using independent and qualified staff, conduct 

surveillance on the EVM system.d No Surveillance on the IRDM EVM system was not conducted and 

IRS does not have staff with sufficient technical skills to 

conduct surveillance.

Sources: GAO analysis of IRDM EVM data, WBSs, and project schedules, as well as OMB and Treasury guidance and GAO-09-3SP. 

a

The American National Standards Institute (ANSI) has a national EVM standard, comprised of 32 

guidelines, that enables agencies to evaluate an EVM system to determine if cost, schedule, and 

technical performance data can be relied on for program management.

b The time-phased budget baseline, against which performance is measured, is formed from the 

performance measurement baseline, which is essentially the resource consumption plan for the 

program. Deviations from the baseline identify areas where management should focus attention. A 

performance measurement baseline represents the cumulative value of a program’s planned work 

over time. This baseline takes into account the program activities that occur in a sequenced order, 

based on finite resources, with budgets representing those resources spread over time.

c An integrated baseline review is an assessment done by program management and contractors to 

verify that the program baseline is adequate and realistically portrays all authorized work according to 

the schedule. For a project of IRDM’s size, guidance from the Department of the Treasury states that 

an independent baseline validation, which appears to be a less rigorous version of an integrated 

baseline review, can be performed.

d

Surveillance is reviewing a contractor’s EVM system to observe ANSI compliance and how well a 

contractor is using its EVM system to manage cost, schedule, and technical performance.

Appendix III: High Level Assessment of the 

Reliability of IRDM’s EVM System
Appendix IV: Comments from the Internal 

Revenue Service

Page 42 GAO-12-59 IRS Management

Appendix IV: Comments from the Internal 

Revenue Service
Appendix IV: Comments from the Internal 

Revenue Service

Page 43 GAO-12-59 IRS Management
Appendix V: GAO Contact and Staff 

Acknowledgments

Page 44 GAO-12-59 IRS Management

Michael Brostek, (202) 512-9110 or brostekm@gao.gov

In addition to the contact named above, Libby Mixon, Assistant Director; 

Laurel Ball; Amy Bowser; Bill Cordrey; Jennifer Echard; Robert Gebhart; 

Paul Middleton; Donna Miller; Sabine Paul; Karen Richey; Stacy Steele; 

and Lindsay Swenson made key contributions to this report.

Appendix V: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments 

(450944)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.