Census 2010: Census at Critical Juncture for Implementing Risk Reduction Strategies
Published: 2008-04-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 2007, the U.S. Census Bureau (Bureau) estimated the 2010 Census would cost $11.5 billion, including $3 billion on automation and technology. At a March hearing, the Department of Commerce (Commerce) stated that the Field Data Collection Automation (FDCA) program was likely to incur significant cost overruns and announced a redesign effort. At that time, GAO designated the 2010 Decennial Census as high risk, citing long-standing concerns in managing information technology (IT) investments and uncertain costs and operations. This testimony is based on past work and work nearing completion, including GAO's observation of the address canvassing dress rehearsal. For IT acquisitions, GAO analyzed system documentation, including deliverables, cost estimates, other acquisitions-related documents, and interviewed Bureau officials and contractors. This testimony describes the implications of redesign for (1) dress rehearsal and decennial operations, (2) IT acquisitions management, and (3) Decennial Census costs.
section_title: What GAO Found
paragraphs: The Decennial Census is at a critical stage in the 2008 Dress Rehearsal, in which the Bureau has its last opportunity to test its plans for 2010 under census-like conditions. Last week Commerce announced significant changes to the FDCA program. It also announced that it expected the cost of the decennial to be up to $3 billion greater than previously estimated. The redesign will have fundamental impacts on the dress rehearsal as well as 2010 Census operations. Changes this late in the decade introduce additional risks, making more important the steps the Bureau can take to manage those risks. The content and timing of dress rehearsal operations must be altered to accommodate the Bureau's design. For example, Commerce has selected an option that calls for the Bureau to drop the use of handheld computers (HHCs) during the nonresponse follow-up operation, and the Bureau may now be unable to fully rehearse a paper-based operation. Additionally, reverting to a paper-based nonresponse follow-up operation presents the Bureau with a wide range of additional challenges, such as arranging for the printing of enumerator forms and testing the systems that will read the data from these forms once completed by enumerators. Given the redesign effort, implementing GAO's recommendations associated with managing the IT acquisitions is as critical as ever. Specifically, the Bureau needs to strengthen its acquisition management capabilities, including finalizing FDCA requirements. Further, it also needs to strengthen its risk management activities, including developing risk mitigation plans for significant risks and improving its executive-level governance of these acquisitions. The Bureau also needs to plan and conduct key tests, including end-to-end testing, to help ensure that decennial systems perform as expected. According to the Bureau, the redesign and related revision of the FDCA program is expected to result in significant increases to the life cycle cost estimate for the 2010 Census. Even without considering the recent expected cost increases announced by the Bureau to accompany the redesign of the FDCA program, the Bureau's cost projections for the 2010 Census revealed an escalating trend from previous censuses. Previously, GAO recommended that the Bureau develop an integrated and comprehensive plan to manage operations. Specifically, to understand and manage the assumptions that drive the cost of the decennial census, GAO recommended, among other actions, that the Bureau annually update the cost of the 2010 Census and conduct sensitivity analysis on the $11.5 billion estimate. However, while the Bureau understands the utility of sensitivity analysis, it has not conducted such an analysis.

section_title: Letter
paragraphs: Thank you for the opportunity to participate in today’s hearing. As you  know, we recently designated the 2010 Decennial Census as a high-risk  area, citing a number of long-standing and emerging challenges facing the  census. These include weaknesses in managing information technology  (IT), operational planning, and cost estimating, as well as uncertainty over  dress rehearsal plans and the ultimate cost of the census.
For 2010, the U.S. Census Bureau (Bureau) planned to make greater use of  technology. Among other things, the Bureau planned to automate some of  its field data collection activities as a way to reduce costs and improve  data quality and operational efficiency. In fact, census workers used new  technology, handheld computers (HHCs), during last spring’s dress  rehearsal operation. The Field Data Collection Automation (FDCA)  program is a key IT acquisition that includes systems, equipment, and  infrastructure for field staff to use in collecting census data for the 2010  Census. Last year, the Bureau had estimated this and other IT acquisitions  would account for about $3 billion of the then-estimated $11.5 billion total  cost of the census. Under the FDCA program the Bureau planned to use  HHCs during operations such as address canvassing, nonresponse follow- up, and census coverage measurement. Last year, during address  canvassing dress rehearsal field activities in California and North Carolina,  both we and Bureau officials observed a number of performance problems  with the HHCs (such as slow and inconsistent data transmissions).
At a March 2008 hearing, the Department of Commerce (Commerce) and  the Bureau stated that the FDCA program was likely to incur significant  cost overruns and said that a redesigning effort to get the Decennial  Census back on track was under way. The Secretary of Commerce  outlined several alternatives for redesigning this central technology  investment, including possibly dropping the HHCs from the nonresponse  follow-up operation. Last week, the Secretary chose to do just that.  Additionally, he decided that the Bureau would reduce deployment of field  technology infrastructure by the contractor, and have the contractor  provide HHCs for address canvassing and develop the information system  for controlling field operations.
As requested, our testimony today will call upon our past work, including  our observation of the use of the HHCs in the address canvassing dress  rehearsal, to provide a description of implications of the redesign for (1)  dress rehearsal and Decennial Census operations, (2) IT acquisitions  management, and (3) Decennial Census costs. Our remarks today are  based primarily on reports that we issued from 2002 through December  2007 on the planning and development of the 2010 Census, as well as the  results of work nearing completion. We visited census test sites in Queens,  New York; several counties in rural south-central Georgia; Austin, Texas;  and the Cheyenne Indian Reservation in South Dakota. During these visits  we observed tests of the address canvassing operation, and we observed  tests of the nonresponse follow-up operation. In May and June 2007, we  observed address canvassing at the 2008 Dress Rehearsal in sites located  in North Carolina and California. For IT acquisitions we analyzed system  documentation, including project plans, deliverables, cost estimates,  earned value management data, other acquisition-related documents, and  we interviewed Bureau officials and contractors.
This work was conducted in accordance with generally accepted  government auditing standards. Those standards require that we plan and  perform the audit to obtain sufficient, appropriate evidence to provide a  reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.
subsections: section_title: Redesign Implications for Decennial Census Operations
paragraphs: The Decennial Census is at a critical stage in the 2008 Dress Rehearsal, in  which the Bureau has its last opportunity to test its plans for 2010 under  census-like conditions. The dress rehearsal features a mock Census Day,  now set for May 1, 2008. Last year at this time, the Bureau carried out a  major dress rehearsal operation—address canvassing—in which the  Bureau updated address lists and collected global positioning coordinates  for mapspots. The largest field operation of the dress rehearsal was to  have begun this month. In this operation (nonresponse follow-up), field  staff were to conduct face-to-face interviews with households that did not  mail back their questionnaires.
Prior to the redesigning effort, the Bureau had already changed its plans  for the dress rehearsal, in part, to focus greater attention on the testing of  technology. In a November 20, 2007, decision memo, the Bureau  announced that it would delay Census Day for the dress rehearsal by 1  month, to May 1, 2008. The Bureau also listed a number of operations it no  longer planned to rehearse, including group quarters enumeration and  census coverage measurement. Also in February 2008, the Bureau  announced that it would remove from the scope of the FDCA program  contract the development of all systems and software associated with the  census coverage measurement operation.
The redesign approach selected by the Secretary will require that the  Bureau quickly develop and test a paper-based nonresponse follow-up  operation. Any paper-based option has its own set of unique issues, such  as setting up operations to support paper field data collection centers and  seeking printing solutions for enumerator forms. Among other issues,  decisions on a printing solution will need to be made soon. Although the  Bureau has carried out paper-based operations before, in some cases they  now involve new procedures and system interfaces that as a result of their  exclusion from the dress rehearsal, will not be tested under census-like  conditions. For nonresponse follow-up in 2010 the Bureau will be using  newly developed systems for integrating responses and controlling  workload. For example, the Bureau will need to rely on a newly developed  system called the Decennial Response Integration System to identify  households that have not returned census forms and for collecting the  results of enumerators conducting nonresponse follow-up person  interviews. Dropping the use of the HHCs for nonresponse follow-up and  reverting to paper for that operation this late in the decade also precludes  nonresponse follow-up from being fully tested in the dress rehearsal.  Under the delayed dress rehearsal this operation was to begin next month,  soon after households in dress rehearsal locations were to return their  census forms. A paper operation requires different training, maps, and  other material to be prepared prior to the operation. The Bureau has  announced no specific plans for conducting field testing of certain key  operations, such as nonresponse follow-up. Without sufficient testing,  operational problems can go undiscovered and the opportunity to improve  operations will be lost.
The redesign’s move from the use of HHCs to a paper-based nonresponse  follow-up operation may limit the Bureau’s ability to reduce follow-up with  persons who are late in returning their census questionnaires. One of the  primary advantages the Bureau cited for using HHCs was the ability, as  late mail returns came in, to remove those addresses from enumerators’  assignments—preventing enumerators from doing unnecessary work.  According to the Bureau, in 2000 enumerators visited over 4 million  households that had returned their census form late. In 2004, the Bureau  tested the capability of an earlier prototype of the HHC to adjust  workloads by identifying late mail returns. We reported in 2007 that based  on these tests it appears that if the Bureau had possessed this capability  during the 2000 Census, it could have eliminated the need to visit nearly  773,000 late-responding households and saved an estimated $22 million  (based on our estimate that a 1 percentage point increase in workload  could add at least $34 million in direct salary, benefits, and travel costs to  the price tag of nonresponse follow-up). The Director of the Census  Bureau stated that he believes that the Bureau can still partially adjust  enumerator workload to recognize late mail returns without the use of  HHCs. To achieve this objective, the Bureau will need to specify the  process it will use and conduct appropriate tests.
The redesign will also affect the 2010 Census address canvassing  operation. The Secretary’s decision to use the HHCs for the 2010 address  canvassing operation means that certain performance issues with the  handheld technology must be addressed promptly. Field staff experienced  difficulties using the technology during the address canvassing dress  rehearsal. For example, workers reported problems with HHCs when  working in large assignment areas during address canvassing. The devices  could not accommodate more than 720 addresses—3 percent of dress  rehearsal assignment areas were larger than that. The amount of data  transmitted and used slowed down the HHCs significantly. Identification  of these problems caused the contractor to create a task team to examine  the issues, and the team recommended improving the end-to-end  performance of the mobile solution by controlling the size of assignment  area data delivered to the HHC both for address canvassing and  nonresponse follow-up operations. One specific recommendation was  limiting the size of assignment areas to 200 total addresses. However, the  redesign effort took another approach that uses laptops and the software  application used for the American Community Survey to collect  information in large assignment areas. It is not yet clear how this   work-around will be carried out. Furthermore, the Bureau will need to  define specific and measurable performance requirements for the HHCs as  we recommended in January 2005.
Another operational issue is the ability of the contractor to accept changes  to its address files after it completes address canvassing updates. This  could preclude the Bureau from conducting “restart/redo” operations for  an area where the address file is discovered to be incorrect. This function  is critical in developing an accurate and complete address list. Without the  ability to update the mailing list for “restart/redo” operations, the Bureau  would consider not mailing census questionnaires to addresses in that  area and instead deliver census forms by hand. This has the potential to  significantly increase costs.
section_title: Redesign Implications for IT Acquisitions
paragraphs: The Bureau still needs to agree upon and finalize requirements for the  FDCA program. In March 2006, we reported that the FDCA project office  had not implemented the full set of acquisition management capabilities  (such as project and acquisition planning and requirements development  and management) that were needed to effectively manage the program.  For example, although the project office had developed baseline  functional requirements for the acquisition, the Bureau had not yet  validated and approved them. Subsequently, in October 2007, we reported  that changes to requirements had been a contributing factor to both cost  increases and schedule delays experienced by the FDCA program. In June  2007, an assessment by an independent contractor of the FDCA program  reported on requirements management problems—much like those we  reported in March 2006. Similar to our recommendation, the independent  assessors recommended that the Bureau immediately stabilize  requirements by defining and refining them. The Bureau has recently made  efforts to further define requirements for the FDCA program, and it has  estimated that the revised requirements will result in significant cost  increases. On January 16, 2008, the Bureau provided the FDCA contractor  with a list of over 400 requirements for the FDCA program to reconcile.  Although some of these new requirements will be dropped based on the  Secretary’s recent decision, many will still need to be addressed to ensure  that FDCA will perform as needed.
Commerce and Bureau officials need to address critical weaknesses in risk  management practices. In October 2007, we reported that the FDCA  project had weaknesses in identifying risks, establishing adequate  mitigation plans, and reporting risk status to executive-level officials. For  example, the FDCA project team had not developed mitigation plans that  were timely or complete nor did it provide regular briefings on risks to  senior executives. The FDCA project team’s failure to report a project’s  risks to executive-level officials reduces the visibility of risks to executives  who should be playing a role in mitigating them.
As of October 2007, in response to the cost and schedule changes, the  Bureau decided to delay certain system functionality for FDCA. As a  result, the operational testing that was to occur during the dress rehearsal  period around May 1, 2008, would not include tests of the full complement  of Decennial Census systems and their functionality. Operational testing  helps verify that systems function as intended in an operational  environment. In late 2007, according to Bureau officials, testing plans for  IT systems were to be finalized in February 2008. Therefore, we  recommended that the Bureau plan and conduct critical testing, including  end-to-end testing of the Decennial Census systems. As of March 2008, the  Bureau still had not developed these test plans. In the recent program  redesign, the Bureau included conducting end-to-end testing. The inability  to perform comprehensive operational testing of all interrelated systems  increases the risk that further cost overruns will occur, that decennial  systems will experience performance shortfalls, or both.
Given the redesigning effort, implementing our recommendations  associated with managing the IT acquisitions is as critical as ever.  Specifically, the Bureau needs to strengthen its acquisition management  capabilities, including finalizing FDCA requirements. Further, it also needs  to strengthen its risk management activities, including developing  adequate risk mitigation plans for significant risks and improving its  executive-level governance of these acquisitions. The Bureau also needs to  plan and conduct key tests, including end-to-end testing, to help ensure  that decennial systems perform as expected.
section_title: Redesign Implications for Decennial Census Life Cycle Costs
paragraphs: Even without considering the recent expected cost increases announced  by the Bureau to accommodate the redesign of the FDCA program, the  Bureau’s cost projections for the 2010 Census revealed an escalating trend  from the 1970 Census. As shown in figure 1, the estimated $11.8 billion  cost (expressed in constant 2010 dollars) of the 2010 Census, before the  FDCA program redesign, represented a more than tenfold increase over  the $1 billion spent on the 1970 Census. The 1970 Census was the first  Census to rely on mailing census forms to households and asking for a  mail return—a major part of the data collection. Although some of the cost  increase could be expected because the number of housing units—and  hence the Bureau’s workload—has gotten larger, the cost growth has far  exceeded the increase in the number of housing units. The Bureau  estimated that the number of housing units for the 2010 Census would  increase by almost 14 percent over Census 2000 levels.
As figure 2 shows, before the FDCA program redesign, the Bureau  estimated that the average cost per housing unit for the 2010 Census was  expected to increase by approximately 26 percent over 2000 levels, from  $69.79 per housing unit to $88.19 per housing unit in constant 2010 dollars.  When the projected cost increase that accompanies the FDCA program  redesign is considered, the average cost per housing unit will increase by  an even greater percentage.
Given the projected increase in spending, it will be imperative that the  Bureau effectively manage the 2010 Census, as the risk exists that the  actual, final cost of the census could be considerably higher than  anticipated. Indeed, this was the case for the 2000 Census, when the  Bureau’s initial cost projections proved to be too low because of such  factors as unforeseen operational problems and changes to the  fundamental design. The Bureau estimated that the 2000 Census would  cost around $5 billion. However, the final price tag for the 2000 Census  was more than $6.5 billion, a 30 percent increase in cost. Large federal  deficits and other fiscal challenges underscore the importance of  managing the cost of the census, while promoting an accurate, timely  census.
We have repeatedly reported that the Bureau would be challenged to  control the cost of the 2010 Census. In January 2004, we reported that  under the Bureau’s approach for reengineering the 2010 Census, the  Bureau might find it difficult to reduce operational risk because  reengineering introduces new risks. To manage the 2010 Census and  contain costs, we recommended that the Bureau develop a  comprehensive, integrated project plan for the 2010 Census that should  include the itemized estimated costs of each component, including a  sensitivity analysis and an explanation of significant changes in the  assumptions on which these costs were based. In response, the Bureau  provided us with the 2010 Census Operations and Systems Plan, dated  August 2007. This plan represented an important step forward at the time.  It included inputs and outputs and described linkages among operations  and systems. However, it did not yet include sensitivity analysis, risk  mitigation plans, a detailed 2010 Census timeline, or itemized estimated  costs of each component. Going forward, it will be important for the  Bureau to update its operations plan.
The assumptions in the fiscal year 2009 President’s Budget life cycle cost  estimate of $11.5 billion may not have included recent productivity data  from last year’s address canvassing dress rehearsal. According to the  Bureau, initially, the cost model assumed productivity for address  canvassing to be 25.6 addresses per hour for urban/suburban areas.  However, results from the address canvassing dress rehearsal showed  productivity of 13.4 addresses per hour for urban/suburban areas. While  the life cycle cost estimate increased slightly to $11.5 billion in the fiscal  year 2009 President’s Budget, these increases were attributed to other  factors and not to lower-than-expected canvassing productivity. Best  practices call for cost model assumptions to be updated as new  information becomes available. We previously reported that the life cycle  cost estimate has not been updated to reflect changes in assumptions. In  July 2006, we testified that the estimate had not been updated to reflect  the results of testing conducted in 2004. As the Bureau updates its  estimate of the life cycle cost annually and as part of the redesigning  effort, it will be important that it reflect changing assumptions for  productivity and hours worked.
section_title: Concluding Observations
paragraphs: Given its size and complexity, carrying out the Decennial Census presents  significant challenges under any circumstances. Late changes in census  plans and operations, long-standing weaknesses in IT acquisition and  contract management, limited capacity for undertaking these critical  management functions, scaling back of dress rehearsal activities, and  uncertainty as to the ultimate cost of the 2010 Census puts the success of  this effort in jeopardy. Managing these risks is critical to the timely  completion of a reliable and cost-effective census. Implementing our  recommendations would help the Bureau effectively manage the myriad of  interrelated operations needed to ensure an accurate and complete count  in 2010 (Bureau officials have agreed with many of our recommendations,  but have not fully implemented them).
The dress rehearsal represents a critical stage in preparing for the 2010  Census. This is the time when the Congress and others should have the  information they need to know how well the design for 2010 is likely to  work, what risks remain, and how those risks will be mitigated. We have  highlighted some of the risks today. Going forward, it will be important for  the Bureau to specify how it will ensure that planned dress rehearsal  operations will be successfully carried out, and how it will provide  assurance that the largest operation—nonresponse follow-up—will be  tested in the absence of a full dress rehearsal. Likewise, the Bureau will  need to establish plans for working around limitations in the technology to  be used in address canvassing operations. It is critical that the Bureau  ensure that the technology for conducting address canvassing is a success.
The Bureau should implement prior recommendations in moving forward.  Contractor-developed IT systems and deliverables need to be closely  monitored to ensure that contractors are performing within budget. As we  have stressed throughout this testimony and in our prior  recommendations, the Bureau needs to practice aggressive project  management and governance over both the IT and non-IT components.  Further, it is essential that the Bureau implement our recommendations  related to information technology. The Bureau must solidify the FDCA  program requirements, strengthen risk management activities, and plan  and conduct critical testing of the Decennial Census systems.
Mr. Chairmen, Census Day is less than 2 years away and address  canvassing is 1 year away. The challenges we highlighted today call for  effective risk mitigation by the U.S. Census Bureau, and careful monitoring  and oversight by the Department of Commerce, the Office of Management  and Budget, the Congress, GAO, and other key stakeholders. As in the  past, we look forward to supporting the committee and subcommittee’s  oversight efforts to promote an accurate and cost-effective census.
Mr. Chairmen, this concludes our statement. We would be glad to answer  any questions you and the committee and subcommittee members may  have.
section_title: Contacts and Acknowledgments
paragraphs: If you have any questions on matters discussed in this testimony, please  contact Mathew Scirè at (202) 512-6806 or sciremj@gao.gov or David A.  Powner at (202) 512-9286 or pownerd@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this testimony. Other key contributors to this testimony  include Signora May, Assistant Director; Mathew Bader; Thomas Beall;  Jeffrey DeMarco; Elizabeth Hosler; Richard Hung; Anne Inserra; Andrea  Levine; Lisa Pearson; Sonya Phillips; Cynthia Scott; Niti Tandon; Jonathan  Ticehurst; Timothy Wexler; and Katherine Wulff.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.