Decennial Census: Additional Actions Could Improve the Census Bureau's Ability to Control Costs for the 2020 Census
Published: 2012-01-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: A complete count of the nations population is an enormous challenge requiring the U.S. Census Bureau (Bureau) to balance requirements for accuracy with the need to control escalating costs. The 2010 Census was the costliest U.S. Census in history at about $13 billion, and was about 56 percent more costly than the $8 billion cost of the 2000 Census (in 2010 dollars). The fundamental challenge facing the Bureau going forward is cost effectively counting a population that is growing steadily larger, more diverse and becoming increasingly difficult to enumerate. As requested, this report assesses (1) the key factors affecting cost growth from the 2000 Census to the 2010 Census; (2) the Bureaus plans for controlling costs for the 2020 Census and what additional steps, if any, could be taken; and (3) the extent to which the Bureaus plans for developing life cycle cost estimates for 2020 are consistent with best practices. The report is based on GAOs analysis of Bureau data and documents as well as interviews with Bureau officials.
section_title: What GAO Found
paragraphs: The average cost to count each housing unit rose from $70 in 2000 to $97 in 2010 (in constant 2010 dollars). While the U.S. Census Bureau (Bureau) made changes to its budget structure from 2000 to 2010, they did not document the changes that would facilitate comparisons over time and cannot identify specific drivers of this cost growth. According to GAOs Cost Estimating and Assessment Guide, an agency can strengthen its ability to control costs by using available cost data to make comparisons over time and identify and quantify trends. The Bureau faces the fundamental challenge of striking a balance between how best to control costs without compromising accuracy. However, the Bureaus inability to identify specific actionable factors affecting past growth will make it difficult for the Bureau to focus its efforts to control costs for the 2020 Census.
The Bureau developed several design alternatives for the 2020 Census that could help reduce costs, but has not identified decision points when executives would review progress and decide whether the Bureau is prepared to move forward from one project phase to another. Office of Management and Budget guidance and previous GAO work support the use of these practices to strengthen an agencys decision making on large-scale projects. Incorporating these practices in its 2020 planning could help the Bureau improve its ability to manage risk to achieve desired cost, schedule and performance outcomes.
The Bureau is taking steps to strengthen its life cycle cost estimates. However, the Bureau has not yet established guidance for developing cost estimates. The Bureau is scheduled to begin work on the 2020 Census estimate in fiscal year 2013 but has limited time to develop guidance. By finalizing such guidance, the Bureau can better ensure that it is developing comprehensive, accurate, and credible estimates for the 2020 Census.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Census Director develop a method to identify and address specific factors that contribute to cost increases, identify decision points, and finalize guidance for the 2020 life cycle cost estimate. The Department of Commerce expressed broad agreement with the overall theme of the report but did not directly comment on the recommendations. It raised concerns about specific aspects of the summary of findings which GAO addressed as appropriate.

section_title: Letter
paragraphs: Although the 2020 Census is still approximately 8 years away, the U.S.  Census Bureau (Bureau) has already begun critical research and testing  activities. The cost of the decennial census has steadily increased during  the past 40 years, in part because of increasing difficulty in accurately  counting a population that is growing larger, more diverse, and  increasingly difficult to enumerate. For example, at about $13 billion, the  2010 Census was the costliest U.S. census in history, and was 56  percent more costly than the $8.1 billion 2000 Census (in constant 2010  dollars). Further, based on past trends, if the growth rate continues  unchecked the census could cost approximately $25 billion in 2020. Thus,  as the Bureau plans for the 2020 Census, it faces the fundamental  challenge of striking a balance between how best to control costs without  compromising accuracy. The Bureau is well aware of this, and its  business plan, which describes its efforts for the early research and  testing phase of the 2020 Census, notes that it is committed to planning  and implementing a 2020 Census that costs less than the approximately  $100 per housing unit that was spent on the 2010 Census.
In response to your request that we review the Bureau’s cost control  efforts for the 2020 Census, this report assesses (1) the key factors  affecting cost growth from the 2000 Census to the 2010 Census; (2) the  Bureau’s plans for controlling costs for the 2020 Census and what  additional steps, if any, could be taken; and (3) the extent to which the  Bureau’s plans for developing life cycle cost estimates for 2020 are  consistent with best practices.
In reviewing key factors affecting cost growth, we reviewed and analyzed  actual cost data from the Bureau’s financial management system for the  2000 Census life cycle and the 2010 Census life cycle.data for inflation and compared broad budget categories from the 2000  and 2010 Censuses; however, limitations in cost data made it impossible   We adjusted the  to conduct comprehensive comparisons below the category level. In  addition, we reviewed Bureau documentation and interviewed officials to  identify factors that could have driven cost growth. To assess the  Bureau’s cost control efforts for the 2020 Census and its plans for  developing life cycle cost estimates, we reviewed available planning  documents for the 2020 Census and interviewed Bureau officials involved  with the planning process. We used GAO’s Cost Estimating and  Assessment Guide to identify best practices for cost estimation. For  more information on our scope and methodology (see app. I).
We conducted this performance audit from December 2010 through  January 2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
On January 6, 2012, the Secretary of Commerce provided written  comments on a draft of this report (see app. II).
subsections: section_title: Background
paragraphs: The cost of the census, in terms of cost for counting each housing unit,  has been escalating over the last several censuses. The average cost for  counting a housing unit increased from about $16 in 1970 to around $97  in 2010 constant dollars (see fig. 1). Meanwhile, the return of census  questionnaires by mail (the primary mode of data collection) declined over  this period from 78 percent in 1970 to 63 percent in 2010. Declining mail  response rates are significant and have led to higher costs because the  mail response rate directly dictates the number of housing units in the  nonresponse follow-up (NRFU) universe. NRFU, where the Bureau  attempts to contact households that did not mail back questionnaires, was  the largest and most costly Bureau field operation in 2000 and 2010 and  has had an impact on overall census costs.
Over the past several censuses, the Bureau has attempted to address the  competing goals of containing costs and improving the quality of census  information, but costs continued to rise in part because external factors,  such as a growing and increasingly diverse population, required the  Bureau to devote more resources in order to ensure a complete count.  The Bureau is assessing various measures of the quality of the 2010  Census. This effort, combined with a better understanding of the specific  sources of cost growth, could help managers make cost control decisions.
Within its financial management system, the Bureau classifies census  costs into eight broad categories and hundreds of projects (see fig. 2).  These broad categories are further subdivided into individual projects that  may be discrete, such as the NRFU operation, which has its costs  captured in a single project line, or several project lines may be  combined, sometimes from multiple categories, to reflect the total cost of  an operation, as is the case with the Local Update of Census Addresses  (LUCA) operation. The 2010 Census costs were concentrated in few  categories.
Planning for the 2020 Census is divided into five phases: (1) options  analysis; (2) research and testing; (3) operational development and  systems testing; (4) supplemental research and testing; and (5) readiness  testing, execution, and closeout. The Bureau has identified a range of  design alternatives for the 2020 Census and will narrow this range over  the census life cycle. During fiscal year 2012, the Bureau will enter the  research and testing phase and intends to develop a preliminary design  that when adjusted for inflation will cost less than the $97 per housing unit  cost of the 2010 Census but will also maintain quality. During the  research and testing phase, the Bureau plans to execute at least 35  research projects to explore how design areas could be modified to  control costs or improve quality. For example, the Bureau will examine  the feasibility of using administrative records, such as Internal Revenue  Service tax records, to collect information from nonresponders and thus  reduce the fieldwork. Other research areas include new response options,  such as the Internet and social networking sites.
The Bureau uses life cycle cost estimates as a starting point for annual  budget formulation and revises the estimates based on appropriations  and updated budget information. As noted in our Cost Estimating and  Assessment Guide, a life cycle cost estimate can be thought of as a  “cradle to grave” approach to managing a program throughout its  duration. However, in our past work, we found that the Bureau’s 2010  Census life cycle cost estimate was not reliable because it lacked  adequate documentation and was not comprehensive, accurate, or  credible. The Bureau may continue to be challenged in developing  reliable life cycle cost estimates for a program as large, costly, and  complex as the census. As part of its planning for 2020, the Bureau has  developed an early life cycle cost estimate based on existing information  and plans to release a full range of life cycle cost estimates in the budget  submission for fiscal year 2015.
section_title: The Bureau Cannot Identify Specific Sources of Cost Growth from 2000 to 2010
subsections: section_title: Field Data Collection Drove Cost Increases, but More Details Are Needed to Help Control Future Costs
paragraphs: Of the Bureau’s eight broad budget categories, field data collection and  its associated support systems accounted for $3.5 billion of the $4.6  billion life cycle cost increase, or 77 percent of the overall cost growth  from 2000 to 2010 (see table 1). This represents a 64 percent growth in  the field data collection category from its 2000 totals, which was the  largest percentage increase of all budget categories. Field data collection  costs include training, labor, and mileage for temporary workers, as well  as the support systems needed to run operations, including rental space  and office equipment for local census offices (LCO). We previously  reported that the field data collection budget category was also the largest  contributor to cost growth from the 1990 Census to the 2000 Census.
The remaining seven budget categories accounted for less than 25  percent of overall cost growth. The automated data collection category  experienced the second largest growth, accounting for 12 percent of  overall cost growth from 2000 to 2010. Expenses in this category were  $547 million more than in 2000, a 42 percent increase. This category  includes data processing activities and related information technology (IT)  system costs. Smaller categories experienced cost growth as well,  including content, questionnaires, and products; census design,  methodology, and evaluation; and census test and dress rehearsal. Other  categories actually experienced cost decreases, including program  development and management.
According to the Bureau, an increased workload—a larger number of  housing units to count—is one of the factors driving up census costs.  This, however, does not fully explain (1) why the cost to count each  housing unit grew at a faster pace than the workload (39 percent increase  to count each housing unit compared to 12 percent increase in workload)  or (2) why component costs, such as data capture systems, experienced  cost increases (see fig. 3).
To more fully understand what is driving up census costs aside from an  increase in workload, it will be important for the Bureau to analyze cost  growth below the category level to determine the specific reasons why  cost per housing unit continues to grow at a faster pace than workload.  Key questions in this regard include, for example, (1) to what extent did  increased labor and gasoline costs contribute to overall increases in field  data collection costs, (2) how did additional use of technology contribute  to field data collection costs, (3) how did increased investments in non- field-related IT systems affect cost growth, and (4) to what extent did the  weak economy in 2010 help the Bureau reduce costs for field operations.  While some cost increases, such as rising gasoline prices, might have  been outside of the Bureau’s direct control, better information on the  sources of census cost growth could enable the Bureau to develop work- arounds and alternatives that could mitigate their impact.
Best practices in GAO’s Cost Estimating and Assessment Guide illustrate  how an agency can strengthen its ability to control costs by using  available cost data to make comparisons over time and identify and  quantify trends. However, the Bureau cannot identify specific sources of  cost growth below broad budget categories from 2000 to 2010 because  the Bureau changed the way it defines projects without creating a  crosswalk that documents the changes over time. As a result, the Bureau  cannot specifically determine where costs are growing.
While it is reasonable for the Bureau to modify its budget structure to  accommodate changes from one decennial to the next, a crosswalk  would have enabled officials to compare costs for specific projects. For  example, for the 2000 Census, 236 projects were identified in the budget.  For the 2010 Census, the Bureau changed its budget structure to more  precisely capture costs, and as a result, the number of projects listed in  the budget increased by almost 400 percent to 1,175 projects. However,  the Bureau created no documentation to facilitate comparison for most  projects in the budget from 2000 to 2010. For example, costs for LUCA  are combined into one project in 2000 data while in 2010 data, LUCA  activities were identified in 11 separate projects (for example, LUCA  processing and LUCA testing). Without documentation explaining what  costs were included in LUCA for the 2000 Census, it is impossible to  accurately compare costs for LUCA between the two decennials and  determine where any cost growth might have occurred.
Further, the Bureau cannot accurately calculate the growth in field  infrastructure costs, if any, from 2000 through 2010 because of a similar  lack of documentation. Although the $2 billion the Bureau spent on its  field infrastructure in 2010—including 12 regional census centers and  almost 500 LCOs used to support field activities—represented a major  investment, the Bureau lacks the information needed to accurately  compare the costs of specific components from one decennial to the next.  Such information would enable the Bureau to more accurately determine  where any significant cost increases occurred and thus better focus its  cost control efforts for the 2020 Census, as well as allow the Bureau to  more precisely determine the potential cost savings of any operational  changes.
Although these structural changes are recent, the absence of  documentation has been a challenge in the past as well. In a prior report  comparing costs for the 1990 and 2000 censuses, we were unable to  compare costs at the project level because of limitations in the available  data and documentation. For the 1990 Census, the Bureau provided  limited cost data by activity and project, so we were not able to attempt  detailed cost comparisons. Moving forward, it will be important for the  Bureau to put a process in place to enhance its ability to identify potential  factors affecting cost growth and, if necessary, target cost control efforts  appropriately.
section_title: Cost Factors Identified by the Bureau Are of Limited Use
paragraphs: Although the Bureau identified five broad factors affecting cost growth,  their ability to help the agency pinpoint and control future costs is limited  because they mainly focus on high-level, generic management  challenges rather than specific census-taking activities on which the  Bureau can assess and take action as appropriate. Additionally, the  Bureau has no data to support how much these factors contributed to  cost growth. The five factors include  1.  the increasing diversity of the population;  2.  the demand for the Census Bureau to strive for improving accuracy  3.  the lack of full public participation in the self-response phase of the  census, requiring the hiring of a large field staff for NRFU;  4.  the failure or challenges with linking major acquisitions, the schedule,  and the budget; and  5.  substantial investments in major national updating of the address  frame just prior to enumeration (2009).
The Bureau plans to use these factors to guide 2020 Census planning  and research efforts. For example, the forthcoming research and testing  phase will focus on the decreasing self-response rate; the linkage of  acquisitions, schedule, and budget; and updates to the address frame.  While these factors, which the Bureau developed through management  experience, likely affected the cost of the census, evaluating the extent to  which specific operations and activities drove up census costs would  provide the Bureau with more actionable information. As one example,  the Bureau identified the demand for improved accuracy as a factor, but  this effort to improve accuracy involved a number of operations aimed at  producing a more complete count, ranging from advertising in different  languages to special enumeration programs aimed at hard-to-count  populations. What is not clear, and will be important for the Bureau to  determine, is how the cost of the special enumeration programs  compared to those for 2000, the extent to which they contributed to the  cost of the 2010 Census, and whether they produced the desired results.
section_title: The Bureau Is Assessing Options for Controlling Field Costs, but Additional Steps Could Be Taken
subsections: section_title: The Bureau Is Considering Design Alternatives to Potentially Reduce Costs for the 2020 Census
paragraphs: The Bureau has developed a range of design alternatives for the 2020  Census aimed at counting each housing unit at a lower cost than in 2010.  The Bureau estimated that if it repeated the design of the 2010 Census,  and assuming real costs grow at the same rate they did between 1990  and 2010, it would cost $151 to count each housing unit—more than a 55  percent increase, compared to 2010. The challenge for the Bureau, as  recognized in its 2020 Census business plan, is striking a balance  between an accurate census, on the one hand, and reducing costs and  managing risks, on the other.
The Bureau’s 2020 design alternatives have potential for containing costs  but at varying degrees of risk for meeting cost, schedule, and  performance goals. The design alternatives focus on options to target  address canvassing, using the Internet and other social media to increase  response rates, and reengineer field and IT infrastructures.shows the current range of 2020 Census design alternatives. According  to the Bureau, the final 2020 design is likely to incorporate both existing  approaches as well as activities that have never been used in the  decennial census, such as a near paperless NRFU.
According to the Bureau, the greater the change to the overall design, the  greater the potential for cost savings. However, greater design changes  also incur greater risk, and further testing will be needed to identify the  risks, costs, and benefits of any new approaches. According to the  Bureau, alternative one has the lowest risk, as it most closely mirrors the  2010 Census design and is not dependent on implementing innovations  such as administrative records and targeted address canvassing. The  remaining alternatives incorporate varying degrees of centralized  infrastructure; address canvassing; and use of administrative records, the  Internet, and social networks. For example, most of the new design  options use administrative records, which could save money by reducing  labor-intensive and costly field operations. Yet, the Bureau has not  previously used administrative records to supplement respondent data on  a national level, so the new approach will present a certain degree of risk  as such things as data quality and access to records must first be  resolved.
section_title: Additional Analysis of 2010 Data Could Improve the Bureau’s Ability to Make Cost Decisions about 2020
paragraphs: The Bureau collects data on the costs of its field operations that are a  potentially valuable source of information to help guide future cost quality  trade-off decisions during the planning process. However, it could make  better use of this information in gaining an understanding of return on  investment for costly census-taking activities, such as address building  and NRFU. According to Office of Management and Budget (OMB)  guidance on benefit-cost analysis, agencies should have a plan for  periodic, results-oriented evaluation of the effectiveness of federal  programs. The guidance also notes that retrospective studies can be  valuable in determining if any corrections need to be made to existing  programs and to improve future estimates of other federal programs. In  addition, our Cost Estimating and Assessment Guide suggests that  agencies should seek the best value solution by gathering data on  alternatives that inform agencies on cost and performance trade-offs.One way agencies can improve their ability to evaluate benefits and costs  is to examine the marginal cost of activities, or the incremental cost of  producing one more unit of output. For the Bureau, this means mining its  performance and cost data to evaluate the effectiveness of its operations  and to identify potential opportunities for improvement.
Although the Bureau has a number of efforts under way within two  initiatives to help guide 2020 planning, only a handful are aimed at  producing return on investment information that enhances its ability to  make decisions on cost quality trade-offs. These initiatives are the 2010  Census Program for Evaluations and Experiments (CPEX) looks back at  2010 operations, and the research and testing phase looks ahead at  potential design alternatives for 2020. According to Bureau officials, of the  more than 100 planned evaluations, assessments, experiments, and  quality profiles in CPEX, a few are designed to produce information  describing the return on investment of census-taking activities, which can  help the Bureau make decisions about cost-quality trade-offs. For  example, 2 planned evaluations will examine potential cost savings for  address canvassing—one looks at the potential cost reduction associated  with targeted address canvassing and the other looks at potential cost  savings associated with automated field data collection of address  canvassing results. Moreover, of planned 2010 CPEX evaluations for  which we have a description, the vast majority will measure aspects of  accuracy or coverage.
The Bureau may be missing opportunities to mine performance data for  information that could help officials increase the efficiency of costly field  operations and could help inform difficult decisions for controlling costs  and maintaining quality. As part of CPEX, the Bureau has planned about  50 assessments of specific enumeration activities and operations, such  as address canvassing and NRFU. These assessments include an  analysis of cost that would be of limited usefulness for informing return on  investment decisions. For example, the assessments will compare  budgeted and actual costs and indicate why an operation was over or  under budget, but will not determine the marginal return for different  enumeration or address-building operations. Information on the marginal  returns on investments could, for example, help the Bureau determine  where to focus cost control efforts. As one example, based on our  analysis of operational data provided by the Bureau for NRFU, we  determined that the marginal cost per questionnaire checked into LCOs  was approximately $1,045 in the final weeks of the operation (see fig. 5).  During this time, the Bureau completed a little over 2,300 questionnaires  or roughly .005 percent of the entire NRFU universe of over 47 million  housing units. This estimate is roughly a $1,000 increase per  questionnaire compared to the first few weeks of the operation, which  began on May 1, when the Bureau completed approximately 39 percent  of the NRFU universe. Thus, it cost the Bureau approximately 17 times  more per questionnaire in the final weeks of NRFU to attempt to obtain  information from nonresponding housing units, units that may have been  contacted as many as six times in person or by phone. More extensive  analyses of these data could help the Bureau determine the extent to  which specific activities contributed to cost growth and help it target  control cost effects without compromising accuracy.
As the Bureau enters the research and testing phase, several planned  projects will yield information that will improve its ability to make decisions  balancing the competing goals of cost and quality. According to the  Bureau, it is essential to conduct research and testing of multiple design  alternatives prior to deciding upon a final census design and technical  solution to ensure that the final census design is effective and works  within the 2020 Census environment. Our review of Bureau planning  documents identified 8 of 35 projects scheduled in the early part of the  decade that will include analyses of costs and benefits. For example, a  project on reducing and improving in-person follow-up operations is  designed to examine the costs and benefits of different contact strategies  and whether these will achieve the goals of the operation. However, most  projects examine the accuracy and quality implications of conducting  enumeration and not cost implications. Without gathering data on cost  during this phase, specifically the potential cost savings that could be  realized with certain alternatives, the Bureau could be making decisions  based on incomplete information on the design alternatives.
The lack of emphasis on cost analyses is consistent with our previous  reports that fundamental reforms will be needed to ensure that the  Bureau’s management, culture, and business practices are aligned with  cost-effective enumeration. According to Bureau officials, previous  decisions about operational changes were based on a priority to improve  quality and were sometimes made without much complete knowledge of  cost implications. As we reported in 2009, the Bureau has not always  used available information to determine the value added of the operation.  For example, the Bureau has the information but has not determined  which of its 11 operations for building its address list provide the best  return on investment in terms of contributing to accuracy and coverage.
section_title: Decision Points Could Help the Bureau Control Costs and Avoid Delays
paragraphs: The Bureau’s planning documents have not clearly identified and defined  decision points that can help avoid cost overruns and schedule delays.  OMB guidance for large projects suggests that agencies develop a  schedule with defined phases, decision points, and an identified decision  authority to evaluate whether an agency should proceed to the next  phase in the investment life cycle. In addition, our previous body of work  on acquisition policies in high-performing organizations includes the best  practice of identifying critical junctures, also known as knowledge or  decision points, in the acquisition cycle and requiring executive-level  oversight at critical junctures. Agencies can use decision points to  determine whether a particular investment is ready to proceed to the next  phase. For example, when moving out of an early phase agencies must  determine if resources—that is, technology and funding—and needs are  matched. The 2020 Census is a complex, costly project with immutable  deadlines. Decision points at key phases of the planning process could  improve the Bureau’s ability to manage risks as well as achieve desired  cost, schedule, and performance outcomes for the decennial.
The Bureau’s 2020 business plan has a high-level preliminary schedule  for the major phases of the decennial that includes, for example, a  yearlong activity at the end of the research and testing phase to  determine and refine initial operational designs. However, the schedule  has no decision points at the end of research and testing or any phase,  as best practices suggest, to determine whether progress was made and  ensure that the agency’s needs for quality and accuracy match the  available resources—that is, technology, design, time, and funding. In  addition, there is no identified executive-level review at any point in the  schedule. Since the research and testing phase is intended to develop a  preliminary design from a range of alternatives, a decision point at the  end of this phase could help the Bureau determine if it has enough  information to support the increased investment necessary to move to the  next phase of development and testing (see fig. 6). At subsequent stages  in the process, decision points could be used to determine that the design  was stable enough to meet operational requirements. Later decision  points could also be used to determine whether a particular design  alternative could be implemented within cost and schedule constraints  while meeting quality targets and maintaining reliability. Absent such an  approach at each phase, the Bureau lacks assurance that it has obtained  the critical technological and design knowledge that best practices call for  to avoid cost overruns, schedule slips, and performance shortfalls going  forward.
According to one Bureau planning memo, cost is one of four categories of  criteria that will be used to evaluate design options for the 2020 Census.  However, the memo does not describe specifically how the cost criterion  will be used to select among design alternatives. For example, the  criterion for cost can be expressed by ranking costs (i.e., least costly to  most costly), weighting costs for different elements, or specifying that  costs fall within a range. We have previously reported that criteria should  be clearly defined, well documented, transparent, and consistently  applied. Neither the Bureau’s strategic plan nor its early business plan,  which outlines and guides the early development of the 2020 Census,  describes criteria or identifies when criteria would be used to select the  design of the 2020 Census.
Bureau officials said they have not established when they will develop  specific evaluation criteria for cost. Further, they acknowledged that  selecting among design alternatives may take place during the research  and testing phase, which begins in fiscal year 2012. In addition, Bureau  officials told us that not all 2020 Census planning memos will be updated  throughout the course of 2020 planning. Therefore, it is unclear how  updates to criteria will be made to this planning memo. As a result, the  Bureau may make decisions to eliminate design alternatives before  clearly documenting how cost criteria will be applied, as well as how the  alternatives will be considered along with the other criteria.
The Bureau’s early cost estimates range from $12.8 billion to $18 billion  for four of the six design alternatives. Because of the wide range of  2020 cost estimates, documenting and consistently using cost as a  criterion when deciding among design alternatives can help the Bureau  control costs. It is important for the Bureau to apply cost in decision  making because the Bureau has not achieved previous goals for  containing costs and made late design changes that proved costly in  previous censuses.
section_title: Without a Process in Place for Developing 2020 Census Life Cycle Cost Estimates, the Bureau Is at Risk of Not Following Best Practices
paragraphs: The Bureau has not yet established policies, procedures, or guidance for  developing the 2020 Census life cycle cost estimate and is at risk of not  following related best practices. The Bureau uses the life cycle cost  estimate as the starting point for the annual budget formulation process  and, according to our Cost Estimating and Assessment Guide, a reliable  cost estimating process is necessary to ensure that cost estimates— particularly for large, complex projects like the 2020 Census—are  comprehensive, well documented, accurate, and credible. Put another  way, reliable cost estimates are essential for a successful census  because they help ensure that the Bureau has adequate funds and that  Congress, the administration, and the Bureau itself can have reliable  information on which to base decisions.
Our guide identifies 12 steps of a high-quality cost estimation process,  including, among other things, determining the estimate’s purpose;  defining the program’s characteristics; clearly defining ground rules and  assumptions; conducting sensitivity, risk, and uncertainty analyses; and  documenting all steps used to develop the estimate. These best  practices, if followed correctly, should produce reliable estimates that  management can use for making informed decisions (see app. IV).
To date, the Bureau has developed a rough-order-of-magnitude estimate,  which covers the four 2020 Census design alternatives that are the most  similar to the 2010 Census design. Bureau officials stated that this was  not an official estimate, but rather a starting point that will be revised and  improved as the Bureau gathers more data in the research and testing  phase. As the Bureau goes forward in its 2020 planning, it will be  important for it to have reliable and accurate cost estimates as it narrows  down design alternatives and moves closer to a final design.
The Bureau’s early 2020 planning documents note that the Bureau  intends to use our cost guide as it develops cost estimates for 2020, and  Bureau officials have stated that its cost estimators would follow best  practices wherever practicable. Nevertheless, the Bureau has not yet  documented how it plans to conduct its cost estimates; and, while officials  stated that they plan on developing more detailed documentation in the  future, they could not provide a specific time when such documents would  be finalized.
Although the 2020 Census is still a number of years away, the timeline for  the Bureau to develop a cost estimation process is growing short. The  Bureau plans to begin work on an official life cycle cost estimate in fiscal  year 2013, and plans to include its initial life cycle cost estimate in its  fiscal year 2015 budget submission covering initiatives from 2015 through  2018. As a result, the Bureau has about a year to establish and finalize a  process for preparing high-quality life cycle cost estimates.
The importance of reliable cost estimates is underscored by the Bureau’s  experience leading up to the 2010 Census, where we found that the  Bureau’s cost estimate lacked detailed documentation on data sources  and significant assumptions and was not comprehensive because it did  not include all costs. Among other weaknesses, we noted that the Bureau  had insufficient policies and procedures for conducting high-quality cost  estimation. Partly as a result, some operations had substantial variances  between their initial cost estimates and their actual costs. Until the Bureau  finalizes its cost estimating policies, procedures, and guidance, the  Bureau runs the risk of again developing unreliable cost estimates for  2020.
section_title: Conclusions
paragraphs: For the Bureau to improve its ability to control the costs of future  censuses without sacrificing accuracy, it will be critical for it to have a  better understanding of the factors affecting cost increases from prior  decennials, as well as how various census-taking activities contributed to  the overall quality of the count. Although the Bureau will gain valuable  insights from its evaluations of the 2010 Census as well as from research  and testing for 2020, this information will only be of limited use in helping  the Bureau develop a complete picture of the key drivers of census costs  and the steps needed to control costs going forward. Therefore, to  improve its capacity to identify cost drivers and effectively target cost  control efforts, it will be important for the Bureau to develop a way to  compare costs for key activities across censuses and assess the  marginal returns of each.
The Bureau has set a clear goal for controlling costs while maintaining  accuracy for the 2020 Census, and has developed a range of design  alternatives aimed at achieving that goal. Given the number of design  alternatives the Bureau is evaluating for the 2020 Census, it will be  important for the Bureau to set explicit decision points for executive-level  review at the end of individual phases to reduce the risk of cost, schedule,  and performance shortfalls. Without clearly defined decision points in its  2020 planning phases, the Bureau may not be able to determine that it is  on track or make the necessary adjustments in its design approach to  achieve a more cost-effective census. Moreover, decision points would  allow the Bureau to determine its readiness to move on to the next phase  in 2020 planning. In conjunction with scheduled decision points, it will be  critical for the Bureau to finalize evaluation criteria that are transparent,  thoroughly documented, and consistently applied to maximize its ability to  control costs for the 2020 Census. Without specifying explicit cost  evaluation criteria for selecting among design alternatives, the Bureau  and stakeholders, such as Congress, cannot accurately consider costs  and may not have assurance that they are on the path to a more efficient  census in 2020.
Cost estimates are necessary tools for major programs because they help  in developing budget requests and efficiently allocating scarce resources.  In a time of constrained budgets, these tools become even more  important. However, cost estimates are technically complex and cost  estimators face challenges in developing estimates for complex programs  such as the 2020 Census. Previously, the Bureau had insufficient policies  and procedures for developing reliable and high-quality cost estimates.  Without clear guidance in place, there is no assurance that the Bureau  will develop life cycle cost estimates for 2020 that are reliable and high- quality and follow best practices.
section_title: Recommendations for Executive Action
paragraphs: To improve the Bureau’s ability to control costs for the 2020 decennial  and balance cost and quality, we recommend that the Secretary of  Commerce direct the Under Secretary of the Economics and Statistics  Administration, as well as the Director of the U.S. Census Bureau, to take  the following four actions:
Develop and document a method to compare costs in 2010 to those in  future decennials, for example, around major activities or investments,  to allow the Bureau to identify and address factors that contribute to  cost increases.
Analyze data from key census-taking activities to determine their  marginal costs and benefits, and use this information to inform  decisions on developing more cost-effective methods.
Identify decision points at the end of each planning phase and assign  decision-making authority at the executive level, as well as consider  adding decision points within phases to determine progress and  readiness to proceed to the next phase.
Finalize how the Bureau will apply cost as an evaluation criterion for  choosing among design alternatives for 2020 and ensure that all  criteria are transparent, well documented, and consistently applied  before alternatives are eliminated.
We have previously recommended that the Secretary of Commerce direct  the Bureau to establish guidance, policies, and procedures for cost  estimation that would meet best practice criteria. To help ensure that the  Bureau produces a reliable and high-quality cost estimate for the 2020  Census, we recommend that the Bureau take the following action:
Finalize guidance, policies, and procedures for cost estimation in  accordance with best practices prior to developing the Bureau’s initial  2020 life cycle cost estimate.
section_title: Agency Comments and Our Evaluation
paragraphs: The Secretary of Commerce provided written comments from the Bureau  on a draft of this report on January 6, 2012. The comments are reprinted  in appendix II. The Department of Commerce expressed broad  agreement with the overall theme of the report but did not directly  comment on the recommendations. It raised concerns about specific  aspects of the summary of findings which GAO addressed as appropriate.
Specifically, the Bureau agrees with the importance of using analysis,  such as assessing marginal returns to help with decision making on  balancing the need to control costs while maintaining accuracy. Moving  forward, it will be important for the Bureau to also recognize that more in- depth understanding of the growth in costs from prior censuses can, in  fact, strengthen its decision-making ability and help it more effectively  target cost control efforts in the future. The Bureau said understanding  the growth in costs from 2000 through 2010 in depth has not been its  highest-priority area for investment of scarce resources. We are sensitive  to existing budget constraints. The fiscal issues facing federal agencies  make it even more imperative for Bureau decision makers to develop and  use actionable information, such as data on the extent to which specific  operations and activities drove up costs, to pinpoint problem areas and  target cost control efforts accordingly.
The Bureau expressed concern that the summary of findings and  conclusions on the highlights page seemed premature and unsupported  by discussions in the full report. In commenting on the first paragraph, the  Bureau stated that it does not believe its inability to identify specific  factors affecting past growth will make it difficult to control costs for the  2020 Census. However, our report concludes that the Bureau’s inability to  identify specific actionable factors will make it difficult for the Bureau to  focus its cost control efforts for the 2020 Census. To help pinpoint  problem areas for controlling census costs, it is important for the Bureau  to have a better understanding of the specific sources of cost growth. This  requires analysis of costs below the broad category level, focusing on  projects that tie directly to major operations and investments. We believe  understanding how the cost of these programs compared to 2000 and the  extent to which they contributed to the cost of the 2010 Census, and  whether they produced the desired results can help with decision making  on areas where there are trade-offs in cost and accuracy. We added  language to the highlights page to reflect the need for analyses to more  effectively target future cost control efforts.
In commenting on the second paragraph of the highlights page, the  Bureau stated that it had not yet received any appropriated funds or had  the opportunity to develop program management efforts for the 2020  Census that would allow the agency to establish formal guidance for  developing cost estimates. However, the paragraph discusses practices  for strengthening agency decision making for large projects rather than  establishing formal guidance for developing cost estimates. OMB  guidance for large projects and our previous body of work on acquisitions  policies in high-performing organizations suggest setting explicit decision  points for executive-level review at the end of individual planning phases  can help reduce the risk of cost, schedule, and performance shortfalls. By  clearly defining decision points in its 2020 planning phases, the Bureau  could better ensure that it is on track or make adjustments in its design  approach earlier in the 2020 planning process to achieve a more cost- effective census. On the issue of having appropriated funds for planning  purposes, we agree that funding for the 2020 Census life cycle did not  officially begin until fiscal year 2012. However, the Bureau includes costs  of early planning for the next census in the final years of the previous  census life cycle (i.e., 2010 appropriations pay for 2020 planning). During  our audit, we interviewed individuals who were planning and developing  the 2020 Census. We also reviewed informational memos, such as the  strategic plan and the business plan issued in 2009 and 2010,  respectively, as guiding documents for the 2020 Census planning effort.  Moreover, during the audit, the Bureau released newly developed and  revised planning documents, such as the updated business plan and  rough-order-of-magnitude estimates. While we made changes to the  second paragraph of the highlights page, these were not made in  response to this comment.
The Bureau commented that it was unclear why the graphic in our  highlights page focused on costs and mail response rate data over time.  We selected mail response rates because, as the Bureau notes, declining  mail response rates are significant and have led to higher costs. For  example, the mail response rate directly dictates the number of housing  units in the NRFU universe. NRFU is the largest and most costly Bureau  field operation and has an impact on overall census costs. We agree with  the Bureau that the declining mail response rate is only one factor leading  to higher census costs. Our report acknowledges other factors that  contribute to higher costs. As such, we made no change to the highlights  page in response to this comment.
The Bureau made a number of technical comments on the body of the  report. The Bureau commented that our report implies that the Bureau  attributed all cost growth over the decades only to population growth. In  fact, our draft report has a section dedicated to the five broad factors the  Bureau identified as affecting cost growth. However, we added clarifying  language to the discussion on workload and census costs to note  workload is one of the factors driving up census costs.
The Bureau commented that the statement that the Bureau cannot  determine areas of cost growth is a sweeping and premature conclusion  given that the 2020 research and testing program just began. The Bureau  stated that its primary focus is to study ways to reduce the cost of the  next census while maintaining quality. While we acknowledge that the  research and testing effort may help identify ways to reduce costs,  coupling that information with specific factors of past cost growth could  strengthen the Bureau’s ability to target cost reduction efforts in the  future. We made no change to the report to address this comment.
The Bureau noted that statements in two areas seemed to be based on  an assessment of how well the Bureau documented and analyzed costs  relative to our Cost Estimating and Assessment Guide. The Bureau states  that the guide was not issued until March 2009 and that the Bureau has  not fully incorporated all those practices into a program as large, costly,  and complex as the census. While it is true that we published the guide in  March 2009, we issued an exposure draft in 2007 and shared a copy of  our guide with the Bureau during our October 2006-June 2008 audit of  Bureau cost estimating practices (GAO-08-554). In fact, the cost guide is  based on long-standing industry and government best practices on cost  estimation followed long before GAO published them in a concise form in  2009. Moreover, in its June 2008 action plan to address GAO  recommendations, the Bureau noted its plan to use the guide, particularly  the 12 steps of a high-quality cost estimating process. In less than 1 year  from now, the Bureau plans to begin work on its official life cycle cost  estimate for the 2020 Census. By not establishing policies, procedures, or  guidance for developing life cycle cost estimates, the Bureau again runs  the risk of developing cost estimates that are not comprehensive,  accurate, or credible. We made no change to the report in response to  this comment.
The Bureau commented that it was unsure why we presented the NRFU  analysis of marginal costs as it was a small percentage of the entire  budget. However, we used this as an example of how such an analysis  may help point to areas for targeting cost reduction efforts or for  modifying the Bureau’s approach to data collection. The analysis does not  imply that the Bureau should ignore the remaining households at the end  of NRFU as the Bureau’s comment states. Instead, it highlights the  importance of considering alternative approaches in order to ensure a  complete and cost-effective enumeration. The more important point is that  it highlights the increasing marginal costs of contacting certain  households at the tail end of the enumeration. We agree that the Bureau  cannot ignore hard-to-contact households. By mining performance data  on the NRFU operation, the Bureau may be in a better position to identify  alternative approaches for the hardest-to-contact households that have  the greatest potential to reduce costs without compromising accuracy. We  made no change to the report to address this comment.
Finally, the Bureau commented that our reported costs for local census  operations were incomplete, so we corrected the number based on  information provided by the Bureau.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to other interested  congressional committees, the Secretary of Commerce, and the Director  of the U.S. Census Bureau. In addition, the report will be available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix V.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: To identify the key factors affecting cost growth from the 2000 Census to  the 2010 Census, we reviewed U.S. Census Bureau (Bureau) strategic  planning documents for 2000 and 2010, Bureau operational and systems  plans for 2000 and 2010, Bureau assessments and evaluations of past  census operations, National Academy of Sciences work on decennial  census costs, and our prior work on implementation of 2000 and 2010  census operations. We assessed the Bureau’s approach to determine  trends in cost data using best practices for cost estimation in GAO’s Cost  Estimating and Assessment Guide. The guide illustrates the importance  of using cost data to understand trends and drivers. In addition, we  interviewed Bureau officials and reviewed agency documentation on  actions taken to determine sources of cost growth between decennials.
To identify sources of cost growth from the 2000 Census and the 2010  Census, we reviewed and analyzed expenditure data on 2000 Census  and 2010 Census life cycle costs from the Bureau’s Commerce Business  System (CBS). CBS is the Bureau’s financial management system and  the official system of record for expenditures. information at two levels of aggregation for the census: budget  categories, which are broad groupings of related items, and budget  projects, which are the lowest level of cost information. To determine the  level of cost growth from the 2000 Census and the 2010 Census, we  developed life cycle totals for each census and life cycle totals for each of  the budget categories within those censuses, comparing their absolute  and percentage growth. We adjusted all monetary data for inflation using  the gross domestic product implicit price deflator. All costs were adjusted  to fiscal year 2010 dollars. In addition, we compared costs after adjusting  for the number of housing units for each census.
We assessed the reliability of the Bureau’s CBS data by reviewing  relevant documentation, interviewing knowledgeable agency officials, and  conducting comparisons with other data sources. We reviewed previous  GAO, Department of Commerce Inspector General, and other  Department of Commerce reports covering the system. We conducted  interviews with Bureau officials who maintain the system at the Bureau  level and its primary users within the Decennial Management Division.
The system is referred to in older GAO reports as the Commerce Administrative  Management System (CAMS). It is the same system, but the name changed over time.
After receiving the cost data covering the 2000 and 2010 censuses, we  compared them to financial management reports provided by Bureau  officials to determine data consistency. We determined that these data  are sufficiently reliable for the purposes of this report.
Our review was subject to some limitations. The budget categories and  budget projects for 2000 and 2010 varied from census to census. We  requested and the Bureau provided recategorized cost data to facilitate  comparison of the 2000 budget categories with 2010 budget categories.  We requested and the Bureau was unable to provide recategorized cost  data to facilitate comparison of 2000 budget projects with 2010 budget  projects. We attempted to compare costs at the budget project level from  2000 to 2010 but were unable to do so for the following reasons: (1) the  Bureau’s budget projects were not consistent from 2000 to 2010, making  it impossible to match projects directly using project descriptions or  project codes; (2) the number of projects increased substantially from  236 in 2000 to 1175 in 2010; and (3) the Bureau was unable to provide us  with any documentation tracking similar projects from the 2000 Census to  the 2010 Census. We attempted to group similar projects in 2000 and  2010 for comparison, but the available project descriptions did not provide  enough information to group 2000 costs with the same precision as 2010.  Therefore, we could not conduct a comparison of groups of projects.
To assess the Bureau’s plans for controlling costs for the 2020 Census  and what additional steps, if any, could be taken, we reviewed available  documentation on 2020 Census planning and 2010 Census evaluations  and assessments, such as 2010 evaluation study plans. We consulted  with GAO staff with expertise in economics to determine the potential for  leveraging available Bureau cost data to better support the Bureau’s  ability to make cost-quality trade-offs. We reviewed Office of Management  and Budget guidance on major acquisitions as well as GAO work on  acquisition best practices to determine whether the use of decision points  could help the Bureau make more informed decisions about census  design that could relate to cost control. Further, we reviewed Bureau  documentation on criteria for selecting among 2020 design alternatives.
We analyzed the marginal cost of conducting nonresponse follow-up  (NRFU)—the costliest field operation in the 2010 Census—to determine  how the Bureau might be able to further use its cost data in planning for  2020. We used Bureau cost and progress data from the 2010 Census to  identify the marginal costs of the NRFU operation in 3-week intervals.  This analysis compared the cost of the operation for that period with the  number of questionnaires checked in to identify return on investment. We  assessed the reliability of the Bureau’s 2010 cost and progress data by  consulting with the Bureau about variables we used and reviewing past  GAO data reliability work that used cost and progress data. The cost and  progress system is a daily management tool used by Bureau officials to  track the work completed of various census operations. It includes  measures of cost (such as field hours or mileage costs) and measures of  work completed (such as questionnaires checked in). Our estimate of the  marginal costs of checking in NRFU questionnaires in the early weeks of  the operation may be somewhat overstated because, for instance, we  included training costs as well as fieldwork costs because training costs  were incurred in the early part of NRFU and those costs were not spread  over the life of the operation. As a result, costs for the early weeks of the  operation could be lower than presented in the graphic.
After developing the marginal costs methodology, we followed up with  agency officials knowledgeable about the data when we had questions  about potential errors or inconsistencies. In addition, we reviewed prior  GAO data reliability work on cost and progress data that examined the  accuracy and completeness of the entry and processing of data. Based  on this work, we determined that the data were sufficiently reliable for  gauging the approximate marginal cost increase per questionnaire  checked in during the final weeks of the NRFU operation.
To assess the extent to which the Bureau’s plans for developing life cycle  cost estimates for 2020 are consistent with best practices, we reviewed  available Bureau documentation on the Bureau’s life cycle cost estimation  processes and procedures. For example, we reviewed documentation  from the Bureau’s rough-order-of-magnitude estimate—an early high- level estimate developed from limited data. We reviewed the guidance  contained in our Cost Estimating and Assessment Guide and our  previous work on census life cycle cost estimates. We also conducted  interviews with knowledgeable Bureau officials and contractor staff and  received a demonstration of new capabilities in the Bureau’s budgeting  tool that will be used for 2020 Census cost estimation.
We conducted this performance audit from December 2010 through  January 2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: 2010 Census Costs Were Concentrated in Certain Budget Categories and Projects
section_title: Appendix IV: The 12 Steps of a High-Quality Cost Estimating Process
subsections: section_title: Step Description 1
section_title: Step Description 6 Obtain data
section_title: Step Description 10
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Signora May, Assistant Director;  Tom Beall; Tim Carr; Eric Charles; Sara Daleski; Dewi Djunaidy; Ron  Fecso; Robert Gebhart; Rich Hung; Kirsten Lauber; Jason Lee; Andrea  Levine; Donna Miller; Stacey Steele; and Jack Wang made key  contributions to this report.