2020 CENSUS

Census Bureau 

Improved the Quality 

of Its Cost Estimation 

but Additional Steps 

Are Needed to 

Ensure Reliability

Report to Congressional Requesters

August 2018

GAO-18-635

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-18-635, a report to 

congressional requesters

August 2018

2020 CENSUS

Census Bureau Improved the Quality of Its Cost 

Estimation but Additional Steps Are Needed to 

Ensure Reliability

What GAO Found

Since 2015, the Census Bureau (Bureau) has made significant progress in 

improving its ability to develop a reliable cost estimate. While improvements 

have been made, the Bureau’s October 2017 cost estimate for the 2020 Census 

does not fully reflect all the characteristics of a reliable estimate. (See figure.) 

Specifically, for the characteristic of being well-documented, GAO found that 

some of the source data either did not support the information described in the 

cost estimate or was not in the files provided for two of its largest field 

operations. In GAO’s assessment of the 2015 version of the 2020 Census cost 

estimate, GAO recommended that the Bureau take steps to ensure that each of 

the characteristics of a reliable cost estimate is met. The Bureau agreed and has 

taken steps, but has not fully implemented this recommendation. 

Extent to Which Census Bureau Cost Estimate for 2020 Census Meets Standards for 

Reliability, 2015 vs. 2017

Source: GAO analysis of U.S. Census Bureau data | GAO 18-635

A reliable cost estimate serves as a tool for program development and oversight, 

helping management make informed decisions. During this review, GAO found 

the Bureau used the cost estimate to inform decision making. 

Factors that contributed to cost fluctuations between the 2015 and 2017 cost 

estimates include: 

• Changes in assumptions. Among other changes, a decrease in the 

assumed rate for self-response from 63.5 percent in 2015 to 60.5 percent in 

2017 increased the cost of collecting responses from nonresponding housing 

units.

• Improved ability to anticipate and quantify risk. In general, contingency 

allocations designed to address the effects of potential risks increased 

overall from $1.3 billion in 2015 to $2.6 billion in 2017. 

• An overall increase in information technology (IT) costs. IT cost 

increases, totaling $1.59 billion, represented almost 50 percent of the total 

cost increase from 2015 to 2017.

View GAO-18-635. For more information, 

contact Robert Goldenkoff at 

(202) 512-2757or goldenkoffr@gao.gov

Why GAO Did This Study

In October 2017, the Department of 

Commerce (Commerce) announced 

that the projected life-cycle cost of the 

2020 Census had climbed to $15.6 

billion, a more than $3 billion (27 

percent) increase over its 2015

estimate. A high-quality, reliable cost 

estimate is a key tool for budgeting, 

planning, and managing the 2020 

Census. Without this capability, the 

Bureau is at risk of experiencing 

program cost overruns, missed 

deadlines, and performance shortfalls.

GAO was asked to evaluate the 

reliability of the Bureau’s life-cycle cost 

estimate. This report evaluates the 

reliability of the Bureau’s revised lifecycle cost estimate for the 2020 

Census and the extent to which the 

Bureau is using it as a management 

tool, and compares the 2015 and 2017 

cost estimates to describe key drivers 

of cost growth. GAO reviewed

documentary and testimonial evidence

from Bureau officials responsible for

developing the 2020 Census cost

estimate and used its cost assessment 

guide (GAO-09-3SP) as criteria.

What GAO Recommends

GAO is not making any new 

recommendations but maintains its 

earlier recommendation—that the 

Secretary of Commerce direct the 

Bureau to take specific steps to ensure 

its cost estimate meets the 

characteristics of a high-quality 

estimate. In its response to this report, 

Commerce generally agreed with the 

findings related to cost estimation 

improvements, but disagreed that the 

cost estimate was not reliable. 

However, until GAO’s recommendation 

is fully implemented the cost estimate 

cannot be considered reliable.
Page i GAO-18-635 2020 Census

Letter 1

Background 2

Census Bureau Has Made Progress but Has Not Taken All the 

Steps Needed to Ensure the Reliability of 2020 Cost Estimate 9

Increased Costs Are Driven by an Assumed Decrease in SelfResponse Rates and Increases in Contingency Funds and IT 

Cost Categories 22

Agency Comments and Our Evaluation 28

Appendix I Objectives, Scope, and Methodology 31

Appendix II Comments from the Department of Commerce 34

Appendix III GAO Contact and Staff Acknowledgments 35

Tables

Table 1: Census Work Breakdown Structure Elements 19

Table 2: Cost Estimation Guidance and Policy Documents 21

Figures

Figure 1: Increases to the 2020 Census Life-Cycle Costs

Estimated by the Census Bureau 6

Figure 2: Census Bureau’s Cost Estimation Process 8

Figure 3: Overview of the Census Bureau’s 2017 Cost Estimate 

Compared to Characteristics of a Reliable Cost Estimate 10

Figure 4: Assessment of Census Bureau’s Cost Estimate for WellDocumented 11

Figure 5: Assessment of Census Bureau’s Cost Estimate for 

Accurate 13

Figure 6: Assessment of Census Bureau’s Cost Estimate for 

Credible 14

Figure 7: Assessment of Census Bureau’s Cost Estimate for 

Comprehensive 18

Figure 8: Assessment of Census Bureau Cost Estimation 

Guidance 22

Contents
Page ii GAO-18-635 2020 Census

Figure 9: Change in 2020 Census Cost Estimate by Work 

Breakdown Structure Category, 2015 vs. 2017 23

Figure 10: 2017 Cost Estimate, IT vs. Non IT Costs 27

Abbreviations

Bureau Census Bureau

CEDCaP Census Enterprise Data Collection and Processing

Commerce Department of Commerce

CQA Census Questionnaire Assistance

ICE independent cost estimate

IT Information Technology

NRFU non-response follow-up

OAM Office of Acquisition Management 

OMB Office of Management and Budget

WBS work breakdown structure

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-18-635 2020 Census

441 G St. N.W.

Washington, DC 20548

August 17, 2018

Congressional Requesters:

The U.S. Census Bureau (Bureau) estimates that the 2020 Census will be 

the most expensive census in our nation’s history, at an estimated cost of 

$15.6 billion after adjusting for inflation. The Bureau faces the challenge 

of cost effectively counting a population that is growing steadily larger, 

more diverse, and increasingly difficult to enumerate with a reengineered 

design that relies in part on automation to locate housing units and count 

the nation’s population.1 In an environment of constrained resources, 

containing costs is imperative for the 2020 Census.

In February 2017, we designated the 2020 Census as a high-risk area, in 

part because of uncertainty over costs and long-standing weaknesses in 

the Bureau’s management of information technology intended to 

automate the census.2 At the time the 2020 Census was designated high 

risk, the life-cycle cost for the 2020 Census was $12.5 billion, a figure 

since revised downward to $12.3 billion.3 However, in October 2017, the 

Department of Commerce (Commerce) announced that it had updated its 

October 2015 life-cycle cost estimate and now projects the life-cycle cost 

of the 2020 Census will be $15.6 billion, a more than $3 billion (27 

percent) increase over its earlier estimate. The higher estimated life-cycle 

cost is due, in part, to the Bureau’s earlier failure to meet best practices 

for a quality cost estimate, a concern we had reported in June 2016.4

 1

Counting the population is required by the U.S. Constitution. Art. I, § 2, cl. 3.

2

GAO, High Risk Series: Progress on Many High Risk Areas, While Substantial Efforts 

Needed on Others, GAO-17-317 (Washington, D.C.: Feb. 15, 2017).

3

The historical life-cycle cost figures for prior decennials as well as the initial estimate for 

2020 provided by Commerce in October 2017 differ slightly from those reported by the 

Bureau previously. According to Commerce documents, the more recently reported 

figures are “inflated to the current 2020 Census time frame (fiscal years 2012 to 2023),”

rather than to constant 2020 dollars as the earlier figures had been. Specifically, since 

October 2017, Commerce and the Bureau have reported the October 2015 estimate for 

the 2020 Census as $12.3 billion; this is slightly different from the $12.5 billion the Bureau 

had initially reported.

4

GAO, 2020 Census: Census Bureau Needs to Improve Its Life-Cycle Cost Estimating 

Process, GAO-16-628 (Washington, D.C.: June 30, 2016).

Letter
Page 2 GAO-18-635 2020 Census

Reliable cost estimates that appropriately account for risks facing an 

agency can help an agency manage large complex activities like the 2020 

Census and can also help Congress provide oversight and make funding 

decisions. Having a realistic estimate of projected costs makes for 

effective resource allocation, and it increases the probability of a 

program’s success.

You requested that we evaluate the reliability of the life-cycle cost 

estimate the Bureau submitted to Congress in October 2017. This report 

(1) evaluates the reliability of the Bureau’s revised life-cycle cost estimate 

for the 2020 Census, and the extent to which the Bureau is using it as a 

management tool; and (2) compares the 2015 and 2017 life-cycle cost 

estimates to describe key drivers of cost growth.

For both objectives, we reviewed documentation related to the cost 

estimate and interviewed Bureau officials responsible for developing the 

2020 Census life-cycle cost estimate. For the first question, we 

interviewed Bureau officials and evaluated whether the Bureau’s cost 

estimate was generated according to best practices of our Cost 

Estimating and Assessment Guide.5 We also assessed the extent to 

which the Bureau is using the cost estimate as a management tool. For 

the second question, we compared cost information included in the 2015 

and 2017 cost estimates. We relied on our cost assessment guide as 

criteria. More information on our scope and methodology can be found in 

appendix I.

We conducted this performance audit from December 2017 to August 

2018 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

A high-quality, reliable cost estimate is a key tool for budgeting, planning, 

and managing the 2020 Census. According to OMB, programs must 

maintain current and well-documented estimates of program costs, and 

 5

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs (Supersedes GAO-07-1134SP), GAO-09-3SP 

(Washington, D.C.: Mar. 2, 2009).

Background
Page 3 GAO-18-635 2020 Census

these estimates must encompass the full life-cycle of the program.6

Among other things, OMB states that generating reliable program cost 

estimates is a critical function necessary to support OMB’s capital 

programming process. Without this capability, agencies are at risk of 

experiencing program cost overruns, missed deadlines, and performance 

shortfalls.

A reliable cost estimate is critical to the success of any federal 

government program. With the information from reliable estimates, 

managers can:

• make informed investment decisions,

• formulate realistic budgets,

• allocate program resources,

• measure program progress,

• proactively correct course when warranted, and

• ensure overall accountability for results.

To be considered reliable, a cost estimate must meet the criteria for each 

of the four characteristics outlined in our Cost Estimating and Assessment 

Guide.7 According to our analysis, a cost estimate is considered reliable if 

the overall assessment ratings for each of the four characteristics are 

substantially or fully met. If any of the characteristics are not met, 

minimally met, or partially met, then the cost estimate does not fully 

reflect the characteristics of a high-quality estimate and cannot be 

considered reliable. Those characteristics are:

• Well-documented: An estimate is thoroughly documented, including 

source data and significance, clearly detailed calculations and results, 

and explanations of why particular methods and references were 

chosen. Data can be traced to their source documents.

 6

Office of Management and Budget, Preparation, Submission, and Execution of the 

Budget, Circular No. A-11 (Washington, D.C.: June 2006); Management of Federal 

Information Resources, Circular No. A-130 Revised (Washington, D.C.: Executive Office 

of the President, Nov. 28, 2000); and Capital Programming Guide: Preparation, 

Submission, and Execution of the Budget, Supplement to Circular A-11, Part 7

(Washington, D.C.: June 2006).

7

GAO-09-3SP.
Page 4 GAO-18-635 2020 Census

• Accurate: An estimate is unbiased, the work is not overly 

conservative or overly optimistic, and is based on an assessment of

most likely costs. Few, if any, mathematical mistakes are present.

• Credible: Any limitations of the analysis because of uncertainty or 

bias surrounding data or assumptions are discussed. Major 

assumptions are varied, and other outcomes are recomputed, to 

determine how sensitive they are to changes in the assumptions. Risk 

and uncertainty analysis is performed to determine the level of risk 

associated with the estimate. The estimate’s results are crosschecked, and an independent cost estimate (ICE) is conducted to see 

whether other estimation methods produce similar results.

• Comprehensive: An estimate has enough detail to ensure that cost 

elements are neither omitted nor double counted. All cost-influencing 

ground rules and assumptions are detailed in the estimate’s 

documentation.

Meeting best practices outlined in our Cost Estimating and Assessment 

Guide for a reliable cost estimate has been a long-standing challenge for 

the Bureau. In 2008 we reported that the 2010 Census cost estimate was 

not reliable because it lacked documentation and was not 

comprehensive, accurate, or credible. For example, in our 2008 report on 

the Bureau’s cost estimation process, Bureau officials were unable to 

provide documentation that supported the assumptions for the initial 2001 

life-cycle cost estimate as well as the updates.8 Consequently, we 

recommended that the Bureau establish guidance, policies, and 

procedures for estimating costs that would meet best practices criteria. 

The Bureau agreed with the recommendation and said at the time that it 

already had efforts underway to improve its future cost estimation 

methods and systems. Moreover, weaknesses in the life-cycle cost 

estimate were one reason we designated the 2010 Census a GAO HighRisk Area in 2008.9

 8

GAO, 2010 Census: Census Bureau Should Take Action to Improve the Credibility and 

Accuracy of Its Cost Estimate for the Decennial Census, GAO-08-554 (Washington, D.C.: 

June 16, 2008).

9

GAO, Information Technology: Significant Problems of Critical Automation Program 

Contribute to Risks Facing 2010 Census, GAO-08-550T (Washington, D.C.: Mar. 5, 2008). 

Other factors contributing to the 2008 high risk designation included long-standing 

weaknesses in the Bureau’s management of information technology and the fact that the 

Bureau delayed the dress rehearsal and dropped several operations. 

Past GAO Work on 

Census Cost Estimation
Page 5 GAO-18-635 2020 Census

In 2012 we reported that, while the Bureau was taking steps to strengthen 

its life-cycle cost estimates, it had not yet established guidance for 

developing cost estimates. We recommended that the Bureau finalize its 

guidance, policies, and procedures for cost estimation in accordance with 

best practices.10 The Bureau agreed with the overall theme of the report 

but did not comment on the recommendation. During this review we found 

that the Bureau took steps to address this recommendation, which is 

discussed later in this report. Such guidance can help to institutionalize 

best practices and ensure consistent processes and operations for 

producing reliable estimates.

In a 2016 report we found that the October 2015 version of the Bureau’s 

life-cycle cost estimate for the 2020 Census was not reliable.11 Overall, 

we reported that the 2020 Census life-cycle cost estimate partially met 

two of the characteristics of a reliable cost estimate (comprehensive and 

accurate) and minimally met the other two (well-documented and 

credible). We recommended that the Bureau take specific steps to ensure 

its cost estimate meets the characteristics of a high-quality estimate. The 

Bureau agreed with this recommendation, and took steps to improve the 

reliability of its cost estimate, which we focus on later in this report. 

Consequently, an unreliable life-cycle cost estimate is one of the reasons 

we designated the 2020 Census a GAO High-Risk Area in 2017.12

In October 2015, the Bureau estimated the cost of the 2020 Census to be 

$12.3 billion. According to the Bureau, the October 2015 version was the 

Bureau’s first attempt to model the life-cycle cost of its planned 2020 

Census, in contrast to its earlier 2011 estimate, which the Bureau said 

was intended to produce an approximation of potential savings and to 

begin developing the methodology for producing decennial life-cycle cost 

estimates covering all phases of the decennial life cycle. To help control 

costs while maintaining accuracy, the Bureau introduced significant 

change to how it conducts the decennial census in 2020. Its planned 

innovations include reengineering how it builds its address list, improving 

self-response by encouraging the use of the Internet and telephone, using 

administrative records to reduce field work, and reengineering field 

 10GAO, Decennial Census: Additional Actions Could Improve the Census Bureau’s Ability 

to Control Costs for the 2020 Census, GAO-12-80 (Washington, D.C.: Jan. 24, 2012). 

11GAO-16-628.

12GAO-17-317.

Development of the 2020 

Cost Estimate
Page 6 GAO-18-635 2020 Census

operations using technology to reduce manual effort and improve 

productivity. In contrast to the estimated $12.3 billion in 2015, the 2020 

Census would cost $17.8 billion in constant 2020 dollars if the Bureau 

repeated the 2010 Census design and methods, according to the 

Bureau’s estimates.

In October 2017, Commerce announced that it had updated the October 

2015 life-cycle cost estimate, projecting the life-cycle cost of the 2020 

Census to be $15.6 billion, an increase of over $3 billion (27 percent) over 

its 2015 estimate. (See figure 1.) In developing the 2017 version of the 

cost estimate, Bureau cost estimators identified cost inputs, their ranges 

for possible outcomes, and overall cost estimating relationships (i.e., 

logical or mathematical formulas, or both). To identify cost inputs and the 

ranges of potential outcomes, the Bureau worked with subject matter 

experts and used historical data to support assumptions and generate 

inputs. The Bureau’s cost estimation team used a software tool to 

generate the cost estimate.13

Figure 1: Increases to the 2020 Census Life-Cycle Costs Estimated by the Census 

Bureau

 13TM1 is an IBM software tool that enables the generation of cost estimates with 

capabilities for detailed modeling of the cost of complex programs comprised of multiple 

products and operations with thousands of variables. The software also supports cost 

model sensitivity and uncertainty analysis around key input variables by defining simple 

statistical distributions around a central estimate for each variable (i.e., minimum, median, 

and maximum), as well as an interface to support quick model re-estimation and Monte 

Carlo simulations. DBiT is an integrated set of applications on the TM1 platform for cost 

modeling and estimation, budget planning, and formulation and execution; as well as for 

analysis and interactive reporting.
Page 7 GAO-18-635 2020 Census

Because cost estimates predict future program costs, uncertainty is 

always associated with them.14 For example, data from the past (such as 

fuel prices) may not always be relevant in the future. Risk and uncertainty 

refer to the fact that because a cost estimate is a forecast, there is always 

a chance that the actual cost will differ from the estimate. One way to 

determine whether a program is realistically budgeted is to perform an 

uncertainty analysis, so that the probability associated with achieving its 

point estimate can be determined, usually relying on simulations such as 

those of Monte Carlo methods. This can be particularly useful in 

portraying the uncertainty implications of various cost estimates.

Consistent with cost estimation practices outlined in our Cost Estimating 

and Assessment Guide, the estimate was compared with two 

independent cost estimates (ICE), developed by Commerce’s Office of 

Acquisition Management (OAM) and the Bureau’s Office of Cost 

Estimation, Analysis, and Assessment. The offices producing the ICEs 

and the cost estimate team worked together to examine the process each 

used, an effort known as the reconciliation process. Through this 

reconciliation, the Bureau identified areas where discrepancies existed

and elements that could require additional review and possible 

improvement.

According to Bureau documentation the estimate will be updated as the 

program meets milestones and to reflect changes in technical or program 

assumptions. Figure 2 details the Bureau’s cost estimation process. OAM 

was involved extensively in the development of the 2017 estimate, an 

increased involvement compared to 2015, according to Bureau officials.

OAM participated in regular review meetings throughout the development 

of the estimate and also developed an independent cost estimate, as 

shown in the figure below.

 14Uncertainty is the indefiniteness about the outcome of a situation. It is assessed in cost 

estimate models to estimate the risk (or probability) that a specific funding level will be 

exceeded.
Page 8 GAO-18-635 2020 Census

Figure 2: Census Bureau’s Cost Estimation Process

End-to-end system testing activities for the 2020 Census are currently 

underway in Providence, Rhode Island. According to the Bureau, 

information collected from the test, such as overall response rates and 

the use of administrative records to inform census records, will inform 

future versions of the life-cycle cost estimate. Some updates from the test 

will be incorporated into the next cost estimate, which will be available in 

the first quarter of the coming fiscal year.
Page 9 GAO-18-635 2020 Census

Since our June 2016 report, in which we reviewed the Bureau’s 2015 

version of the cost estimate, the Bureau has made significant progress.15

For example, the Bureau has put into place a work breakdown structure 

(WBS) that defines the work, products, activities, and resources 

necessary to accomplish the 2020 Census and is standardized for use in 

budget planning, operational planning, and cost estimation. However, the 

Bureau’s October 2017 cost estimate for the 2020 Census does not fully 

reflect characteristics of a high-quality estimate as described in our Cost 

Estimating and Assessment Guide and cannot be considered reliable.16

Our Cost Estimating and Assessment Guide describes best practices for 

developing reliable cost estimates. For our reporting needs, we collapsed 

these best practices into four characteristics for sound cost estimating—

comprehensive, well-documented, accurate, and credible—and identified 

specific best practices for each characteristic. To be considered reliable, 

an organization must meet or substantially meet each characteristic. Our 

review found the Bureau met or substantially met three out of the four 

characteristics of a reliable cost estimate, while it partially met one 

characteristic: well-documented. When compared to the October 2015 

estimate, the 2017 estimate shows considerable improvement. (See 

figure 3 below.)

 15GAO-16-628.

16GAO-09-3SP.

Census Bureau Has 

Made Progress but 

Has Not Taken All the 

Steps Needed to 

Ensure the Reliability 

of 2020 Cost 

Estimate
Page 10 GAO-18-635 2020 Census

Figure 3: Overview of the Census Bureau’s 2017 Cost Estimate Compared to 

Characteristics of a Reliable Cost Estimate

Note: Not Met – Census Bureau provided no evidence that satisfies the criterion; Minimally Met –

Census Bureau provided evidence that satisfies a small portion of the criterion; Partially Met –

Census Bureau provided evidence that satisfies about half of the criterion; Substantially Met –

Census Bureau provided evidence that satisfies a large portion of the criterion; and Met – Census 

Bureau provided complete evidence that satisfies the entire criterion.

Cost estimates are considered valid if they are well-documented to the 

point they can be easily repeated or updated and can be traced to original 

sources through auditing, according to best practices. The Bureau only 

partially met the criteria for well-documented, as set forth in our Cost 

Estimating and Assessment Guide. A cost estimate that does not fully 

meet the criteria for well-documented cannot be used by management to 

make informed and effective implementation decisions.

The well-documented characteristic comprises five best practices. The 

Bureau substantially met two out of five best practices (as shown in figure 

4). First, the estimate describes in sufficient detail the calculations 

performed and the estimating methodology used to derive each element’s 

cost, and the cost estimate had been reviewed by management. Since 

cost estimates can inform key decisions and budget requests, it is vital 

that management review and understand how the estimate was 

developed, including risks associated with the underlying data and 

methods.

Well-Documented
Page 11 GAO-18-635 2020 Census

Figure 4: Assessment of Census Bureau’s Cost Estimate for Well-Documented

The cost estimate only partially met three best practices for the 

characteristic of being well-documented. In general, some documentation 

was missing, inconsistent, or difficult to understand. First, we found that 

source data did not always support the information described in the basis 

of estimate document or could not be found in the files provided for two of 

the Bureau’s largest field operations: Address Canvassing and NonResponse Follow-Up (NRFU).17 For example, the cost estimate 

documentation referred to actual data from the 2010 Census and 

information obtained from experts as sources for address canvassing 

rework rates.18 However, the folder source documents provided as 

 17The basis of estimate document describes in detail the scope of the estimate, the cost 

approaches and ground rules, the data sources, main assumptions, and cost estimating 

relationships.

18The address canvassing rework rate refers to the percentage of housing units that will 

need to be revisited during initial in-field data collection. The Bureau’s address canvassing 

operation updates its address list and maps, which are the foundation of the decennial 

census, through data collection efforts conducted both in the office and through in-field 

visits to address locations.
Page 12 GAO-18-635 2020 Census

support for the basis of estimate did not include this information. Next, in 

several cases, we could not replicate calculations, such as for mileage 

costs, using the description provided. Lastly, we found that some of the 

cost elements did not trace clearly to supporting spreadsheets and 

assumption documents.

Failure to document an estimate in enough detail makes it more difficult to 

replicate calculations, or to detect possible errors in the estimate; reduces 

transparency of the estimation process; and can undermine the ability to 

use the information to improve future cost estimates or even to reconcile 

the estimate with another independent cost estimate. The Bureau told us 

it would continue to make improvements to ensure the estimate is welldocumented. For the estimate to be considered well-documented, the 

Bureau will need to address these issues. 

An accurate cost estimate supports measurement of program progress by 

providing unbiased and correct data, which can help management ensure 

accountability for scheduled results. We found the Bureau’s cost estimate 

substantially met the criteria for accuracy. As shown in figure 5, and in 

line with best practices outlined in our Cost Estimating and Assessment 

Guide, the estimate was not overly optimistic; appeared to be free of

errors; was based on historical data or input from subject matter experts; 

and, according to Bureau officials, is updated regularly as information 

becomes available.

Accurate
Page 13 GAO-18-635 2020 Census

Figure 5: Assessment of Census Bureau’s Cost Estimate for Accurate

The Bureau can enhance the accuracy of their estimate by increasing the 

level of detail included in the documentation, such as detail on specific 

inflation indices used, and by monitoring actual costs against estimates. 

We identified areas for improvement, which, according to Bureau officials, 

will be addressed as part of its ongoing efforts. For example, while the 

basis of estimate document describes different inflation indexes, it was 

not clear exactly which indexes were applied to the various cost elements 

in the estimate. Also, evidence of how variances between estimated costs 

and actual expenses would be tracked over time was not available at the 

time of our analysis. Tools to track variance enable management to 

measure progress against planned outcomes. Bureau officials stated that 

they already have systems in place that can be adapted for tracking 

estimated and actual costs.

All estimates include a certain amount of informed judgment about the 

future. Assumptions made at the start of a program can turn out to be 

inaccurate. Credible cost estimates identify limitations due to uncertainty 

or bias surrounding data or assumptions, and control for these 

uncertainties by identifying and quantifying cost elements that represent 

Credible
Page 14 GAO-18-635 2020 Census

the most risk. We found that the Bureau’s cost estimate substantially met 

the criteria for credible, as shown in figure 6 below.

Figure 6: Assessment of Census Bureau’s Cost Estimate for Credible

The Bureau’s cost estimate clearly identifies risks and uncertainties, and 

describes approaches taken to mitigate them. In line with best practices 

outlined in our Cost Estimating and Assessment Guide, the Bureau did 

the following:

• Sensitivity analysis. The Bureau conducted sensitivity analysis to 

identify possible changes to estimated costs for the 2020 Census 

based on varying major assumptions, parameters, and data inputs.19

For example, the Bureau calculated the likely cost implications for a 

range of possible response rates to identify a range of projected costs 

and to calculate appropriate reserves for risk. Bureau officials stated 

 19Sensitivity analysis first identifies key elements that drive cost and their associated 

assumptions and then calculates the estimate’s sensitivity to changes in the underlying 

assumptions.
Page 15 GAO-18-635 2020 Census

that they also identified the estimate input parameters that contributed 

the most to estimate uncertainty.

• Risk and uncertainty analysis. A cost estimate is a forecast, and as 

such, there is always a chance that the actual cost will differ from the 

estimate. Uncertainty is the indefiniteness about the outcome of a 

situation. Uncertainty is assessed in cost estimate models to estimate 

the risk (or probability) that a specific funding level will be exceeded. 

We found the Bureau performed an uncertainty analysis on a portion 

of the estimate to determine whether estimated costs were realistic 

and to establish the probability of achieving projections outlined in the 

estimate. The Bureau used a combination of modeling based on 

Monte Carlo analysis and allocations of funding for risks.20 The Monte 

Carlo simulation was performed on a portion of the estimate to 

account for uncertainty around various operational parameters for 

which a range of outcomes was possible, including Internet response 

rates and the extent to which data collection issues might be resolved 

using administrative records. To account for the inherent uncertainty 

of assumptions included within the life-cycle cost estimate, the Bureau 

added funding to the cost estimate totaling approximately $292 million 

to account for risks based on the results of the Monte Carlo analysis.

For other risks, such as acquisition lead time and the possibility of 

delays in information technology (IT) development, contingency 

funding was added to the estimate to reflect the potential cost of 

resolving these issues, through use of a backup system or an 

alternative approach. These are described as “special risks” in the 

Bureau’s basis of estimate, and total approximately $171 million.

Based on additional sensitivity analysis, the Bureau added 

approximately $965 million to the cost estimate to reflect discrete risks 

outlined in the risk register as well as those associated with (1) 

variability in self-response rates, (2) the effect of fluctuations in the 

size and wage rate of the temporary workforce on the cost of field 

operations, and (3) the potential need to reduce the enumerator-to-

 20The Monte Carlo simulation randomly samples parameters from a probability distribution 

for each variable (using parameters that define a statistical distribution, such as minimum, 

most likely, and maximum) and then uses those values to calculate an output distribution, 

preserving the logical and mathematical relationship among inputs and outputs. Typically, 

after thousands of simulation runs, an output distribution is generated as a result of the 

analysis.
Page 16 GAO-18-635 2020 Census

manager staffing ratio in case expected efficiencies in field operations 

are not realized.

In addition to these provisions, the Secretary of Commerce added a 

contingency amount of about $1.2 billion to account for what the 

Bureau refers to as unknown-unknowns. Bureau documentation 

states that conducting a decennial census is an extremely complex, 

high-risk operation. In order to mitigate some of the risk, contingency 

funding must be available to initiate ad hoc activities necessary to 

overcome unforeseen issues. According to Bureau documentation 

these include such risks as natural disasters or cyber-attacks. The 

Bureau provides a description of how the general risk contingency is 

calculated. However, this description does not clearly link calculated 

amounts to the risks themselves.

In our June 2016 report we reported the Bureau had not properly 

accounted for risk and recommended the Bureau, in part; improve 

control over how risk and uncertainty are accounted for. We continue 

to believe the prior recommendation from our June 2016 report 

remains valid and should be addressed: that the Bureau properly 

account for risk in the 2020 Census cost estimate, among other 

things. As such, risks need to be linked to the $1.2 billion general risk 

contingency fund.

• Independent cost estimate. According to best practices outlined in 

our Cost Estimating and Assessment Guide, an independent cost 

estimate should be performed to determine whether alternate 

estimate approaches produce similar results. The Bureau compared 

their estimate with two independent cost estimates, developed by 

Commerce’s Office of Acquisition Management and the Bureau’s 

Office of Cost Estimation and Assessment. As part of their process for 

finalizing the cost estimate, Bureau officials reconciled differences 

between the estimates in discussions with the two offices, resulting in 

more conservative assumptions by the Bureau around risk and 

uncertainty in both cases.

In addition to implementing our recommendation to properly account 

for risk, going forward, while the Bureau substantially met the 

credibility characteristic it will be important for them to also integrate 

regular cross-checks of methodology into their cost estimation 
Page 17 GAO-18-635 2020 Census

process.21 In our analysis we observed that no specific cross-checks 

of cost methodology were performed. According to the Bureau, crosschecks were not performed because the Bureau considered the 

independent cost estimates as overall cross-checks on the reliability 

of their methodology and did not conduct additional cross-checks. The 

main purpose of cross-checking is to determine whether alternative 

methods for specific cost elements within the cost estimate could 

produce similar results. An independent cost estimate, though 

important for the credibility of an estimate, does not fulfill the same 

function as a targeted cross-check of individual elements.22

Comprehensive estimates have enough detail to ensure that cost 

elements are neither omitted nor double-counted, all cost-influencing 

assumptions are detailed in the estimate’s documentation, and a work 

breakdown structure is defined.23 Our analysis of the 2017 cost estimate 

demonstrates improvement over the 2015 cost estimate when the 

Bureau’s cost estimate only partially met the criteria for comprehensive.24

We found the Bureau met or substantially met all four best practices for 

the comprehensive characteristic, as shown in figure 7. For example, all 

life-cycle costs are included in the estimate along with a complete 

description of the 2020 Census program and current schedule. We also 

 21To cross-check is to apply a different method for cost estimation for cost drivers. If the 

alternative method produces similar results, then confidence in the estimate increases 

leading to greater credibility. 

22Cross-checks and independent cost estimates are two different techniques for ensuring 

the credibility of a cost estimate. Cross-checking is performed by the original cost 

estimating team, on individual cost elements, to validate their primary estimating 

methodologies. Independent cost estimates are typically performed by organizations 

higher in the decision-making process than the office performing the baseline estimate. 

They provide an independent view of expected program costs that tests the program 

office’s estimate for reasonableness.

23A WBS defines in detail the work necessary to accomplish a program’s objectives. It 

describes the products and activities necessary to complete the program and provides a 

basis for identifying resources and tasks needed. 

24If the cost estimate is not comprehensive, then it cannot fully meet the well-documented, 

accurate, or credible best practice characteristics. For instance, if the cost estimate is 

missing some cost elements, then the documentation will be incomplete, the estimate will 

be inaccurate, and the result will not be credible due to the potential for underestimating 

costs and the lack of a full risk and uncertainty analysis.

Comprehensive
Page 18 GAO-18-635 2020 Census

found that the Bureau substantially met criteria for documenting cost 

influencing ground rules and assumptions.

Figure 7: Assessment of Census Bureau’s Cost Estimate for Comprehensive

A standardized WBS (as detailed in table 1) with supporting dictionary 

outlines the major work of the program and describes the activities and 

deliverables at the project level where costs are tracked. In 2016, the 

Bureau’s WBS did not contain sufficient detail and we found significant 

differences in the presentation of the work between sources. In 2017, 

based on our review of Bureau documentation and interviews with Bureau 

officials, we found that the WBS is standardized and cost elements are 

presented in detail. The WBS is a necessary program management tool 

because it provides a basic framework for a variety of related tasks like 

estimating costs, developing schedules, identifying resources, 

determining where risks may occur, and providing the means for 

measuring program status.
Page 19 GAO-18-635 2020 Census

Table 1: Census Work Breakdown Structure Elements

Program Management Policies, processes, and control functions for planning 

and implementing the 2020 Census.

Census Survey 

and Engineering

Scientific, technical, and managerial efforts needed to 

evolve, verify, or deploy and support a census.

Frame Activities related to the development of a geospatial 

frame (addresses and maps) to serve as the universe for 

enumeration activities.

Response Deliverables and activities required to access, maintain, 

and process the response data.

Published Data Activities related to preparing and disseminating 

publishable data. 

Test and Evaluation To ensure all systems and activities meet the technical 

and operational needs of the Bureau.

Infrastructure Administrative functions, service, logistics, information 

technology, and operational support.

Source: GAO analysis of U.S. Census Bureau documents. | GAO-18-635.

Although the Bureau’s updated life-cycle cost estimate reflects three of 

the four characteristics of a reliable cost estimate, we are not making any 

new recommendations to the Bureau in this report. We continue to 

believe the prior recommendation, made in 2016, remains relevant: that 

the Secretary of Commerce ensure that the Bureau finalizes the steps 

needed to fully meet the characteristics of a high-quality estimate, most 

notably in the well-documented area. The Bureau told us it has used our 

best practices for cost estimation to develop their cost estimate, and will 

focus on those best practices that require attention moving forward. 

Without a reliable cost estimate, the Bureau is limited in its ability to make 

informed decisions about program resources and to effectively measure 

progress against operational objectives.

OMB, in its guidance for preparing and executing agency budgets, cites 

that credible cost estimates are vital for sound management decision 

making and for any program or capital project to succeed.25 A welldeveloped cost estimate serves as a tool for program development and 

oversight, supporting management to make informed decisions. 

 25Office of Management and Budget, Capital Programming Guide: Supplement to Circular 

A-11, Planning, Budgeting and Acquisition of Capital Assets. (Washington, D.C.: July 

2016). 

Life-Cycle Cost Estimate 

Is Used by Management 

to Inform Decisions
Page 20 GAO-18-635 2020 Census

According to the Bureau, the 2020 Census cost estimate is used as a 

management tool to guide decision making. Bureau officials stated the 

cost estimate is used to examine the cost impact of program changes. 

For example, the cost estimate served as the basis for the fiscal year 

2019 funding request developed by the Bureau. The Bureau also said it 

used the 2020 Census life-cycle cost estimate to establish cost controls 

during budget formulation activities and to monitor spending levels for

fiscal year 2019 activities. According to the Bureau, as detailed 

operational and implementation plans are defined, the 2020 Census lifecycle cost estimate has been and will continue to be used to support 

ongoing “what-if” analyses in determining the cost impacts of design 

decisions. Specifically, using the cost estimate to model the impact of 

changes on overall cost, the Bureau adjusted the scope of the Census 

Enterprise Data Collection and Processing (CEDCaP) operation.26

The processes for developing and updating estimates are designed to 

inform management about program progress and the use of program 

resources, supporting cost-driven planning efforts and well-informed 

decision making. Our work has identified a number of best practices for 

use in developing guidance related to cost estimation and analysis that 

are the basis of effective program cost estimating and should result in 

reliable and valid cost estimates that management can use for making 

informed decisions.27

In 2012 we reported that the Bureau had not yet established guidance for 

developing cost estimates.28 We recommended that the Bureau establish 

guidance, policies, and procedures for developing cost estimates that 

would meet best practice criteria. The Bureau agreed with the theme of 

the report but did not specifically agree with the recommendation. 

Moreover, in June 2016, we also reported that the cost estimation team 

did not record how and why it changed assumptions that were provided to 

it and did not document the sources of all data it used.29 The 

documentation of these changes to assumptions did not happen because 

 26CEDCap is an enterprise-wide initiative intended to deliver a system-of-systems to 

support all of the Bureau’s survey data collection and processing functions.

27GAO-09-3SP.

28GAO-12-80.

29GAO-16-628.

Census Bureau Guidance 

to Develop Cost Estimates 

Meets Best Practices
Page 21 GAO-18-635 2020 Census

the Bureau lacked written guidance and procedures for the cost 

estimation team to follow. During this review we found the Bureau has 

since established reliable guidance, processes, and policies for 

developing cost estimates and managing the cost estimation process. 

The following documents, shown in table 2, establish roles and 

responsibilities for oversight and approval of cost estimation processes, 

provide a detailed description of the steps taken to produce a high-quality 

cost estimate, and clarify the process for updating the cost estimate and 

associated documents over the life of a project.

Table 2: Cost Estimation Guidance and Policy Documents 

Title Description

Cost Estimation and Analysis 

Guidance

Establishes the overall authority, requirements, 

activities, roles, and responsibilities for cost estimation 

and supporting analysis specifically within the 

Decennial Census Programs Directorate.

Cost Center of Excellence 

Charter

Charter for the core cost estimation body responsible 

for coordinating, facilitating, and supporting cost 

estimation and analysis activities across the 2020 

Census Program. 

Cost Estimate and Analysis 

Process 

Step-by-step guidance and a framework for how the 

2020 Census Program conducts cost estimation and 

analysis.

2020 Life-Cycle Cost Estimate 

Version Control Plan 

Establishes a disciplined approach to cost estimate 

updates, changes, and releases.

Source: GAO analysis of U.S. Census Bureau documents. | GAO-18-635.

The Decennial Census Program’s Cost Estimate and Analysis Process, 

which provides a detailed description of the steps taken to produce a 

high-quality estimate, is reliable as it met the criteria for 8 steps and 

substantially met the criteria for 4 steps of the 12 best steps outlined in 

our Cost Estimating and Assessment Guide, as shown below in figure 8. 

To avoid cost overruns and to support high performance, it will be 

important for the Bureau to abide by their newly developed policies and 

guidance and continue to use the life-cycle cost estimate as a 

management tool.
Page 22 GAO-18-635 2020 Census

Figure 8: Assessment of Census Bureau Cost Estimation Guidance

The 2017 life-cycle cost estimate includes significantly higher costs than 

those included in the 2015 estimate. In 2015, the Bureau estimated that 

they could conduct the operation at a cost of $12.3 billion in constant 

2020 dollars. The Bureau’s latest cost estimate, announced in October 

2017, reflects the same design, but at an expected cost of $15.6 billion. 

Figure 9 below shows the change in cost by WBS category for 2015 and 

2017. The largest increases occurred in the Response, Managerial 

Contingency, and Census/Survey Engineering categories. Increased 

costs of $1.3 billion in the response category (costs related to collecting, 

maintaining, and processing survey response data) were in part due to 

reduced assumptions for self-response rates, leading to increases in the 

amount of data collected in the field, which is more costly to the Bureau. 

Contingency allocations increased overall from $1.35 billion in 2015 to 

$2.6 billion in 2017, as the Bureau gained a greater understanding of 

risks facing the 2020 Census. Increases of $838 million in the 

Census/Survey Engineering category were due mainly to the cost of an IT 

contract for integrating decennial survey systems that was not included in 

Increased Costs Are 

Driven by an 

Assumed Decrease in 

Self-Response Rates 

and Increases in 

Contingency Funds 

and IT Cost 

Categories
Page 23 GAO-18-635 2020 Census

the 2015 cost estimate. Bureau officials attribute a decrease of $551 

million in estimated costs for Program Management to changes in the 

categorization of costs associated with risks: In the 2017 version of the 

estimate, estimated costs related to program risks were allocated to their 

corresponding WBS element.

Figure 9: Change in 2020 Census Cost Estimate by Work Breakdown Structure Category, 2015 vs. 2017
Page 24 GAO-18-635 2020 Census

More generally, factors that contributed to cost fluctuations between the 

2015 and 2017 cost estimates include:

30

• changes in assumptions for census operations,

• improved ability to anticipate and quantify risk,

• an overall increase in IT costs, and

• more defined contract requirements.

Several assumptions for the implementation of the 2020 Census have 

changed since the 2015 cost estimate. Some assumptions contributing to 

cost changes, mainly in the Response (related to collecting and 

processing response data) and Frame (the mapping and collecting 

addresses to frame enumeration activities) categories, include the 

following:

• Self-response rates. Changes in assumptions for expected selfresponse rates contributed to increases in the response category, as 

the assumed rate decreased from 63.5 percent in 2015 to 60.5 

percent in 2017, thereby increasing the anticipated percentage and 

associated cost of nonresponse follow-up.

31 When the Bureau does 

not receive responses by mail, phone, or Internet, census 

enumerators visit each nonresponding household to obtain data. 

Thus, reduced self-response rates lead to increases in the amount of 

data collected in the field, which is more costly to the Bureau. Bureau 

officials attributed this decrease to a forecasted reduction in Internet 

response due to added authentication steps at log in and the 

elimination of the function allowing users to save their responses and 

return later to complete the survey.

• Productivity rates. The productivity of enumerators collecting data 

for NRFU is another variable in the cost estimate that was updated, 

contributing to cost increases in the response category. Expected 

productivity rates for NRFU decreased from the 2015 estimate of 4 

attempts per hour to 2.9. According to Bureau documentation, this 

 30Many of these factors are interrelated and their contribution to increases in certain cost 

categories may overlap. As one example, IT increases overlap with the Bureau’s ability to 

define their contract requirements. Though we recognize the interconnection between 

these factors, we describe these separately for ease of explanation.

31NRFU refers to enumerators personally visiting households that did not respond to the

census through mail, phone or Internet. 

Changes in Assumptions
Page 25 GAO-18-635 2020 Census

more conservative estimate is based on historical data, rather than 

research and test data.

• In-office address canvassing rates. The Bureau will not go door-todoor to conduct in-field address canvassing across the country to 

update address and map information for every housing unit, as it has 

in prior decennial censuses. Rather, some areas would only need a 

review of their address and map information using computer imagery 

and third-party data sources—what the Bureau calls “in-office”

address canvassing procedures. However, in March 2017, citing 

budget uncertainty the Bureau decided to discontinue one of the 

phases of in-office review address canvassing for the 2020 Census. 

The cancellation of that phase of in-office review is expected to 

increase the number of housing units canvassed in-field by 5 percent 

(from 25 to 30 percent of all canvassed housing units). In-field 

canvassing is more labor intensive compared to in office procedures. 

The 2017 version of the cost estimate reflects this increase in 

workload for in-field address canvassing, though overall changes in 

estimated costs for the Frame category, of which Address Canvassing 

is a part, were minimal.

• Staffing. Updated analysis resulted in changes to several staffing 

assumptions, which resulted in decreases across WBS categories. 

Changes included reduced pay rates for field data collection staff 

based on current labor market conditions and reductions in the length 

of staff engagement.

In general, contingency allocations increased overall from $1.35 billion in 

2015 to $2.6 billion in 2017. This increase in contingency can be 

attributed, in part, to the Bureau gaining a clearer understanding of risk 

and uncertainty in the 2020 Census as it approaches. The Bureau 

developed some of its contingency based on proven risk management 

techniques, including Monte Carlo analysis and allocated funding for 

known risk scenarios. The 2017 estimate includes close to $1.4 billion in 

estimated costs for these risks, almost three times the amount included in 

the 2015 estimate. The basis of estimate contains detail on the various 

risks and the process for calculating the associated contingency. The 

2017 version also includes a contingency amount of $1.2 billion for 

general risks, or unknown-unknowns, such as natural disasters and 

cyber-attacks.

Contingency amounts were reallocated within the WBS to more closely 

reflect the nature of the risk: Bureau officials attribute a decrease from the 

Anticipation of Risk
Page 26 GAO-18-635 2020 Census

2015 estimate of $551 million in estimated costs for program 

management to changes in the categorization of costs associated with 

risks. Officials stated that, in 2015, discrete program risks were previously 

consolidated as program management costs. In 2017, these discrete 

costs were reallocated to associate risks with the appropriate WBS 

element. For example, contingency amounts related to the likelihood of 

achieving a certain response rate previously included in the program 

management work breakdown category are now a part of the “response”

work breakdown category.

Increases in IT costs, totaling $1.59 billion, represented almost 50 percent 

of the total cost increase from 2015 to 2017. The total share of IT costs as 

a percentage of total census costs increased from 28 percent in 2015 to 

32 percent in 2017, or from $3.41 billion to approximately $5 billion. 

Increases in IT costs are spread across seven cost categories. Figure 10 

shows the IT and non-IT cost by WBS for the 2017 cost estimate. IT costs 

in infrastructure, response data, and census/survey WBSs account for the 

majority of the approximately $5 billion.

Increased IT Costs
Page 27 GAO-18-635 2020 Census

Figure 10: 2017 Cost Estimate, IT vs. Non IT Costs

The Bureau’s October 2015 cost estimate included IT costs for, among 

other things, system engineering, test and evaluation, and infrastructure, 

as well as for a portion of the Census Enterprise Data Collection and 

Processing (CEDCaP) program. The 2017 estimated IT cost increases 

were due, in large part, to the Bureau (1) updating the cost estimate for 

CEDCaP; (2) including an estimate for technical integration services that 

contributed to increases in the Census and Survey Engineering category; 

and (3) updating costs related to other major contracts (such as mobile 

device as a service, field IT services, and payroll systems).
Page 28 GAO-18-635 2020 Census

Bureau documents described an overall improvement in the Bureau’s 

ability to define and specify contract requirements. This resulted in 

updated estimates for several contracts, including for the Census 

Questionnaire Assistance (CQA) contract.32 Assumptions regarding call 

volume to the CQA were increased by 5 percent to account for expected 

response by phone after the elimination of the option to save Internet 

responses and return to complete the form later. The Bureau also cited 

updated cost data and the results of reconciliation with independent cost 

estimates as factors contributing to the increased costs of other major 

contracts, including for the procurement of data collection devices.

The Secretary of Commerce provided comments on a draft of this report 

on August 2, 2018. The comments are reprinted in appendix II. The 

Department of Commerce generally agreed with our findings regarding 

the improvements the Census Bureau has made in its cost estimates. 

However, Commerce did not agree with our assessment that the 

Bureau’s 2017 lifecycle cost estimate is “not reliable.” Commerce noted 

that it had conducted two independent cost analyses and was satisfied 

that the cost estimate was reliable. The Bureau also provided technical 

comments that we incorporated, as appropriate.

We maintain that, to be considered reliable, a cost estimate must meet or 

substantially meet the criteria for each of the four characteristics outlined 

in our Cost Estimating and Assessment Guide. These characteristics are 

derived from measures consistently applied by cost estimating 

organizations throughout the federal government and industry and are 

considered best practices for the development of reliable cost estimates. 

Without a reliable cost estimate, the Bureau is limited in its ability to make 

informed decisions about program resources and to effectively measure 

progress against operational objectives. Thus, while the Bureau has 

made considerable progress in all four of the characteristics, it has only 

partially met the criteria for the characteristic of being well-documented. 

Until the Bureau meets or substantially meets the criteria for this 

characteristic, the cost estimate cannot be considered reliable. 

 32CQA has two primary functions: to provide 1) questionnaire assistance by telephone and 

email for respondents by answering questions about the census in general and regarding 

specific items on the census form, and 2) an option for respondents to complete a census 

interview over the telephone. 

Contract Requirements

Agency Comments 

and Our Evaluation
Page 29 GAO-18-635 2020 Census

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of the report to the 

appropriate congressional committees, the Secretary of Commerce, the 

Under Secretary of Economic Affairs, the Acting Director of the U.S. 

Census Bureau, and other interested parties. In addition, this report is 

available at no charge on the GAO website at http://gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. GAO staff who made key contributions to this 

report are listed in appendix III.

Robert Goldenkoff

Director, Strategic Issues
Page 30 GAO-18-635 2020 Census

List of Requesters

The Honorable Claire McCaskill 

Ranking Member

Committee on Homeland Security and Governmental Affairs 

United States Senate

The Honorable Gary Peters 

Ranking Member 

Subcommittee on Federal Spending Oversight and Emergency 

Management 

Committee on Homeland Security and Governmental Affairs 

United States Senate

The Honorable Elijah E. Cummings 

Ranking Member

Committee on Oversight and Government Reform

House of Representatives

The Honorable John A. Culberson 

Chairman

The Honorable José Serrano 

Ranking Member

Subcommittee on Commerce, Justice, Science, and Related Agencies 

Committee on Appropriations 

House of Representatives
Appendix I: Objectives, Scope, and 

Methodology

Page 31 GAO-18-635 2020 Census

The purpose of our review was to evaluate the reliability of the Census 

Bureau’s (Bureau) life-cycle cost estimate using our Cost Estimating and 

Assessment Guide.1 We reviewed (1) the extent to which the Bureau’s 

life-cycle cost estimate and associated guidance met our best practices 

for cost estimation using documentation and information obtained in 

discussions with the Bureau related to the 2020 life-cycle cost estimate 

and (2) compared the 2015 and 2017 life-cycle cost estimates to describe 

key drivers of cost growth. For both objectives we reviewed 

documentation from the Bureau on the 2020 life-cycle cost estimate and 

interviewed Bureau and Department of Commerce officials.

For the first objective, we relied on our Cost Estimating and Assessment 

Guide as criteria. Our cost specialists assessed measures consistently 

applied by cost-estimating organizations throughout the federal 

government and industry and considered best-practices for developing 

reliable cost estimates. We analyzed the cost estimating practices used 

by the Bureau against these best practices and evaluated them in four 

categories: comprehensive, well-documented, accurate, and credible.

• Comprehensive. The cost estimate should include both government 

and contractor costs of the program over its full life-cycle, from 

inception of the program through design, development, deployment, 

and operation and maintenance to retirement of the program. It should 

also completely define the program, reflect the current schedule, and 

be technically reasonable. Comprehensive cost estimates should be 

structured in sufficient detail to ensure that cost elements are neither 

omitted nor double counted. Specifically, the cost estimate should be 

based on a product-oriented work breakdown structure (WBS) that 

allows a program to track cost and schedule by defined deliverables, 

such as hardware or software components. Finally, where information 

is limited and judgments are made, the cost estimate should 

document all cost-influencing assumptions.

• Well-documented. A good cost estimate—while taking the form of a 

single number—is supported by detailed documentation that 

describes how it was derived and how the expected funding will be 

spent in order to achieve a given objective. Therefore, the 

documentation should capture in writing such things as the source 

data used, the calculations performed and their results, and the 

estimating methodology used to derive each WBS element’s cost. 

 1

GAO-09-3SP.

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 32 GAO-18-635 2020 Census

Moreover, this information should be captured in such a way that the 

data used to derive the estimate can be traced back to, and verified 

against, their sources so that the estimate can be easily replicated 

and updated. The documentation should also discuss the technical 

baseline description and how the data were normalized. Finally, the 

documentation should include evidence that the cost estimate was 

reviewed and accepted by management.

• Accurate. The cost estimate should provide for results that are 

unbiased, and it should not be overly conservative or optimistic. An 

estimate is accurate when it is based on an assessment of most likely 

costs; adjusted properly for inflation; and contains few, if any, minor 

mistakes. In addition, a cost estimate should be updated regularly to 

reflect significant changes in the program—such as when schedules 

or other assumptions change—and actual costs, so that it is always 

reflecting current status. During the update process, variances 

between planned and actual costs should be documented, explained, 

and reviewed. Among other things, the estimate should be grounded 

in a historical record of cost estimating and actual experiences on 

other comparable programs.

• Credible. The cost estimate should discuss any limitations of the 

analysis because of uncertainty or biases surrounding data or 

assumptions. Major assumptions should be varied, and other 

outcomes recomputed to determine how sensitive they are to changes 

in the assumptions. Risk and uncertainty analysis should be 

performed to determine the level of risk associated with the estimate. 

Further, the estimate’s cost drivers should be cross-checked, and an 

independent cost estimate conducted by a group outside the acquiring 

organization should be developed to determine whether other 

estimating methods produce similar results.

If any of the characteristics are not met, minimally met, or partially met, 

then the cost estimate does not fully reflect the characteristics of a highquality estimate and cannot be considered reliable.

We also analyzed the Bureau’s cost estimation and analysis guidance 

and evaluated them against a 12-step process outlined in our Cost 

Estimation and Assessment Guide. A high-quality cost estimating process 

integrates the following:

1. Define estimate’s purpose.

2. Develop estimating plan.

3. Define program characteristics.
Appendix I: Objectives, Scope, and 

Methodology

Page 33 GAO-18-635 2020 Census

4. Determine estimating structure.

5. Identify ground rules and assumptions.

6. Obtain data.

7. Develop point estimate and compare it to an independent cost 

estimate.

8. Conduct sensitivity analysis.

9. Conduct risk and uncertainty analysis.

10. Document the estimate.

11. Present estimate to management for approval.

12. Update the estimate to reflect actual costs and changes.

These 12 steps, when followed correctly, should result in reliable and 

valid cost estimates that management can use for making informed 

decisions. If any of the steps in the Bureau’s process do not meet, 

minimally meet, or partially meet the 12 steps, then the cost estimate 

guidance does not fully reflect best practices for developing a high-quality 

estimate and cannot be considered reliable.

Lastly, to describe key drivers of cost growth, we compared cost 

information included in the 2015 and 2017 cost estimates. We analyzed 

both summary and detailed cost information to assess key changes in 

totals overall, by WBS category, and by information technology (IT) vs. 

Non-IT costs. We used this analysis in conjunction with information 

received from the Bureau during interviews and through document 

transfers to describe overall changes in the cost estimate from 2015 to 

2017.

We conducted this performance audit from December 2017 to August 

2018 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Comments from the Department 

of Commerce

Page 34 GAO-18-635 2020 Census

Appendix II: Comments from the Department 

of Commerce
Appendix III: GAO Contact and Staff 

Acknowledgments

Page 35 GAO-18-635 2020 Census

Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov

In addition to the contact named above, Lisa Pearson (Assistant Director), 

Karen Cassidy (Analyst in Charge), Brian Bothwell, Jackie Chapin, Ann 

Czapiewski, Jason Lee, Ty Mitchell, Kayla Robinson, and Tim Wexler 

made significant contributions to this report.

Appendix III: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff

Acknowledgments

(102518)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to https://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, https://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at https://www.gao.gov.

Contact:

Website: https://www.gao.gov/fraudnet/fraudnet.htm

Automated answering system: (800) 424-5454 or (202) 512-7700

Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.