2010 Census: Planning and Testing Activities Are Making Progress
Released: 2006-03-01T00:00:00

Letter

Background

Thorough and comprehensive planning is crucial to the success of any large, long-term project, especially one with the cost, complexity, and high stakes of the decennial census. Indeed, the Bureau’s past experience has shown that the lack of proper planning can increase the costs and risks of downstream operations.

Past experience has also underscored the importance of strong oversight of the census to (1) inform congressional decision making on budgetary and operational matters, (2) raise Congress’s confidence that the Bureau has chosen an optimum design and will manage operations and control costs effectively, and (3) help ensure the progress the Bureau has made thus far in refining, planning, and testing census-taking activities, continues as the Bureau shifts into the operational phases of the decennial.

Given the escalating cost of the census in an era of serious national fiscal challenges, oversight will be particularly important. Bureau officials estimate the total life-cycle cost of the 2010 Census will be around $11.3 billion, which would make it the most expensive census in our country’s history, even after adjusting for inflation.

Although some cost growth can be expected, in part, because the number of housing units—and hence the Bureau’s workload—has grown, the cost escalation has far exceeded the housing unit increase. The Bureau estimates that the number of housing units for the 2010 Census will increase by 10 percent over 2000 Census levels; meanwhile, the average cost per housing unit for 2010 is expected to increase by approximately 29 percent from 2000 levels (from $56 to $72), nearly five and a half times greater than the $13 it cost to count each household in 1970 (see fig. 1).

A key reason for the increasing cost of the census is that because of various societal trends such as concerns over personal privacy, more non- English speakers, and more people residing in makeshift and other nontraditional living arrangements, the Bureau is finding it increasingly difficult to locate people and get them to participate in the census. As a result, the Bureau needs to spend more money simply to achieve the accuracy of earlier enumerations. This can be seen, for example, in the rising cost of securing public participation in the census. During the 1990 Census, the Bureau spent an average of $0.88 per housing unit (in 2000 dollars) to market the census and was able to rely on a pro-bono advertising campaign. The response rate was 65 percent. For the 2000 Census, recognizing that extra effort would be needed to motivate participation, the Bureau used a paid advertising campaign developed by a consortium of private-sector advertising agencies. It cost an average of $3.19 per housing unit (in 2000 dollars) and achieved a response rate of 64 percent. As the Bureau plans for 2010, maintaining cost effectiveness will be one of the single greatest challenges confronting the agency.

The Bureau’s Preparations for 2010 Are Progressing but Certain Challenges Will Need to Be Addressed

The Bureau’s preparations for the 2010 Census appear to be further along than at a similar point during the planning cycle for the 2000 Census. For example, the fundamental design of the 2010 Census has the potential to contain costs and improve coverage and accuracy, and the Bureau’s planning process for 2010 is generally more thorough than was the case for the 2000 Census. At the same time, the 2004 test and, to date, the 2006 test, have identified areas where improvements are needed. Uncovering trouble spots is an important objective of any test, thus it is not surprising, and, in fact, should be expected and commended that problems were found. Moreover, the Bureau has taken steps to resolve the issues that have surfaced. Remaining activities in the 2006 test, and the 2008 Dress Rehearsal, will help determine the effectiveness of the Bureau’s efforts.

The Design of the 2010 Census Shows Promise

The Bureau developed a design for the 2010 Census early in the decade, and Congress has been supportive of the Bureau’s approach. The situation 10 years ago was vastly different. In testimony before Congress in late 1995, we expressed concern that Congress and the Bureau had not agreed on the fundamental design and budget of the census, and that the longer this situation continued, the opportunity for a well-planned census would be lost and the greater the risk that hundreds of millions of dollars would be spent inefficiently.

Key features of the design of the 2010 Census include the following: Enhancing procedures for building its address list, known as the Master Address File, and its associated geographic information system, called the Topologically Integrated Geographic Encoding and Referencing (TIGER)® database; Replacing the census long-form questionnaire with the American Community Survey (ACS); and Conducting a short-form-only decennial census supported by early research and testing.

Also noteworthy is the fact that for the 2010 Census, the Bureau plans to make the most extensive use of contractors in its history, turning to the private sector to supply a number of different mission-critical functions, including the Bureau’s nationwide data processing activities, and improvements to the Master Address File and TIGER. The Bureau estimates that of the $11.3 billion total cost of the census, around $1.9 billion (approximately 17 percent) will be spent via its seven largest contracts which include information technology systems, advertising, and the leasing of local census offices.

The Bureau is relying more heavily on contractors because it recognizes it needs to look outside the agency to obtain the expertise and services essential for a successful enumeration. That said, the Bureau’s contracting efforts during the 2000 Census did not always go smoothly, and it will be important for Bureau management to focus on its procurement activities to help ensure the 2010 contractors fulfill the Bureau’s expectations. Our companion testimony at today’s hearing provides greater detail on two of the Bureau’s information technology contracts.

In concept, the Bureau’s approach has the potential to achieve its principal goals for the 2010 Census which include (1) increasing the relevance and timeliness of data, (2) reducing operational risk, (3) increasing coverage and accuracy, and (4) containing costs. However, some aspects of the design, including the use of technology that has never been employed for the decennial, as well as the heavy reliance on contractors, introduce new risk. This is not inappropriate as the need to secure a complete count and addressing problems with past censuses call for bold new initiatives that entail risk. What will be important is how effectively the Bureau manages those risks.

The 2010 Planning Process Is Generally More Rigorous Than for Past Efforts

Another sign of progress can be found in the thoroughness of the Bureau’s planning process where the Bureau has taken several positive steps to correct problems it encountered when planning past censuses. For example, early in the decade, senior Bureau staff considered various goals for the 2010 Census and articulated a design to achieve those goals. Moreover, staff with operational experience in the census participated in the 2010 design process. According to Bureau officials, this was a departure from the 2000 planning effort when Bureau staff with little operational experience played key roles in the design process, which resulted in impractical reform ideas that could not be implemented.

At the same time, the Bureau’s planning process could benefit from an overall business or project plan that (1) includes milestones for completing key activities; (2) itemizes the estimated cost of each component; (3) articulates a clear system of coordination among project components; and (4) translates key goals into measurable, operational terms to provide meaningful guidance for planning and measuring progress. Some, but not all of this information is available in various documents, but one would need to piece it together. Noting the importance of this information to inform congressional decision-making and oversight of the census, as well as to improve the Bureau’s planning process, in our January 2004 report, we recommended that the Bureau combine this information into a single, comprehensive document. The Bureau disagreed with the recommendation although it said it would develop such a plan nonetheless and provide it to GAO, Congress, and other stakeholders. The Bureau has not yet issued this document.

Address File and Map Improvements Could Help Resolve Past Problems if Operational Challenges Are Resolved

A complete and accurate address list is the cornerstone of a successful census because it identifies all households that are to receive a census questionnaire, and serves as the control mechanism for following up with households that fail to respond. Although the Bureau went to great lengths to build a complete and accurate Master Address File for the 2000 Census, of the 116 million housing units contained in the database, the Bureau estimates it incorrectly included 2.3 million housing units and missed another 2.7 million housing units. In light of these and other problems, the Bureau concluded that enhancements to the Master Address File and TIGER were necessary to make census data more complete and accurate.

In the preliminary results of our ongoing work on enhancements to the Master Address File and TIGER, we found that the Bureau has developed procedures to help resolve each of the broad categories of problems experienced in 2000 including addresses that were duplicated, missed, deleted, and incorrectly located on a map (a problem known as geocoding error). The Bureau has several ongoing evaluations that should provide valuable information on the effectiveness of these procedures.

The Bureau is also taking steps to improve the accuracy of the TIGER maps which, among other benefits, should help prevent geocoding errors. In June 2002, the Bureau awarded an 8-year contract, in excess of $200 million intended to, among other tasks, correct in TIGER the location of every street, boundary, and other map feature so that coordinates are aligned with their true geographic locations. According to the Bureau, the contractor completed this work for 250 counties in 2003, 602 counties in 2004, and 623 counties in 2005. Furthermore, the contractor plans to deliver the remaining 1,758 county maps between 2006 and 2008.

However, based on this time line, it appears that several hundred county TIGER maps will not be updated in time for the Local Update of Census Addresses (LUCA) program, through which the Bureau gives local and tribal government officials the opportunity to review and suggest corrections to the address lists and maps for their jurisdictions. LUCA is to begin in July 2007 when, according to the current schedule, the Bureau will still have 368 counties to update in 2008 alone. These counties will not have the most current maps to review but will instead be given the most recent maps the Bureau has available. According to the Bureau, some of the maps have been updated for the American Community Survey, but others have not been updated since the 2000 Census, which could affect the quality of a local government’s review. The Bureau is aware of the overlapping schedules, but told us that it needs to start LUCA in 2007 in order to complete the operation in time for address canvassing.

LUCA is an example of how the Bureau partners with external entities, tapping into their knowledge of local populations and housing conditions in order to secure a more complete count. In 1994, Congress required the Bureau to develop a local address review program to give local and tribal governments greater input into the Bureau’s address list development process.

When the Bureau conducted LUCA for the 2000 Census, the results were mixed. In our 1999 congressional testimony, we noted that many local governments said they were satisfied with specific aspects of the materials and assistance the Bureau provided to them. At the same time, LUCA may have stretched the resources of local governments, and participation in the program could have been better.

The census schedule will also be a challenge for an operation called address canvassing, where census workers are to walk every street in the country, verifying addresses and updating maps as necessary. The Bureau has allotted 6 weeks to verify the nation’s inventory of 116 million housing units. This translates into a completion rate of over 2.75 million housing units every day. The challenge in maintaining this schedule can be seen in the fact that for the 2000 Census, it took the Bureau 18 weeks just to canvass “city-style” address areas, which are localities where the U.S. Postal Service uses house-number and street-name addresses for most mail delivery.

The Unreliability of Mobile Computing Devices Has Been Problematic

Of particular concern is the previous unreliability of the MCDs the Bureau plans to use for its address canvassing and nonresponse follow-up operations (see fig. 2).

For address canvassing, the MCDs are to be loaded with address information and maps; for nonresponse follow-up, they will be used in lieu of paper questionnaires and maps to collect household information. The MCDs are also equipped with global positioning system (GPS) receivers, a satellite-based navigational system to help enumerators locate street addresses and to collect coordinates for each structure in their assignment area. Bureau officials expect the MCDs will help improve the cost- effectiveness of the census by allowing it to eliminate millions of paper questionnaires and maps, improve the quality of address data, and update enumerators’ nonresponse follow-up workload on a daily basis.

The move from paper to digital was a very positive step. At the same time, rigorous testing is essential to assess their durability, functionality, and that enumerators are able to use them. The MCDs were first evaluated for nonresponse follow-up as part of the 2004 Census Test, and for address canvassing in 2005 as part of the 2006 Census Test. The Bureau will use MCDs next month for nonresponse follow-up in the 2006 test.

In both our prior and ongoing work, we found the test results have been mixed. On the one hand, the census workers we observed had little difficulty using the MCDs. For example, address canvassers we interviewed said the electronic maps were accurate and that they were able to find their assignment areas with relative ease. On the other hand, the reliability of the MCDs proved troublesome during the 2004 and to date, the 2006 test. For example, in 2004, the MCDs experienced transmission problems, memory overloads, and difficulties with a mapping feature--all of which added inefficiencies to the nonresponse follow-up operation.

The Bureau is using MCDs made by a different manufacturer for the 2006 test which resolved some of these problems, but other difficulties emerged during address canvassing. For example, the device was slow to pull up and exit address registers, accept the data entered by the canvassers, and link map locations to addresses for multi-unit structures. Furthermore, the MCDs would sometimes lock up, requiring canvassers to reboot them.

Canvassers also found it difficult to transmit an address and map location that needed to be deleted from the master list. The Bureau was unable to fix this problem so canvassers had to return to the local census office where technicians dealt with the problem. The reliability of the GPS was also problematic. Some workers had problems receiving a signal, and when a signal was available, it was sometimes slow to locate assignment areas and correct map locations.

According to the Bureau, these problems reduced the productivity of the canvassers, and the Bureau stopped the operation 10 days after it was scheduled to finish. Even with the extension, however, the Bureau was unable to complete the job, leaving census blocks in both Austin and on the Cheyenne Indian Reservation unverified.

According to the Bureau, the problems were caused by unstable software and insufficient memory. The Bureau delayed the start of address canvassing for a month at both test sites to troubleshoot the MCDs. However, it was unable to fix all the problems and decided to move forward with the test.

The MCDs will be evaluated again next month as part of the 2006 Census Test and we will be on-site to assess the extent to which the Bureau has fixed the MCD problems. However, even if the MCDs prove to be more reliable, questions remain for the future. The Bureau has acknowledged that the MCD’s performance is an issue, but believes it will be addressed as part of its contract for the Field Data Collection Automation (FDCA) program, which is aimed at automating the Bureau’s field data collection efforts, and is scheduled to be awarded later this month (the MCDs used for the 2006 test are off-the-shelf purchases that were customized by the Bureau).

As a result, the 2008 Dress Rehearsal will be the first time the entire system—including the contractor’s MCD—will be tested under conditions that are as close as possible to the actual census. If new problems emerge, little time will be left to develop and test any refinements.

Our field observations also suggest that the training of census workers could be improved to help ensure they follow proper procedures. Failure to do so could affect the reliability of census data. During the 2004 test, for example, we observed enumerators who did not read the coverage and race/ethnicity question exactly as worded, and did not properly use flashcards the Bureau had developed that were designed to help respondents answer specific questions. During the address canvassing operation for the 2006 test, we observed workers who were not properly verifying addresses, or were unsure of what to do when they happened upon dwellings such as duplex housing units. In our past work, we recommended that the Bureau take a more strategic approach to training, and that local census offices include in their instruction special modules covering the unique living arrangements that might be prevalent in that particular jurisdiction.

The Bureau acknowledged that the shortcomings we identified require improvement, and indicated that for the 2006 test, it will enhance training to reinforce the procedural requirements. The Bureau also intends to incorporate additional training to prepare enumerators to handle realistic situations encountered in their work. As part of our field work for the 2006 test, we will review the improvements the Bureau made to its training procedures.

Succession Planning and Management Will Be Important in the Years Ahead

If the operational challenges of conducting a census were not daunting enough, the Bureau faces the additional challenge of a possible brain drain. In our June 2005 report, we noted that the Bureau has projected that 45 percent of its workforce will be eligible to retire by 2010. The Bureau has long benefited from its core group of managers and experienced staff who developed their expertise over several census cycles; their institutional knowledge is critical to keeping the census on track. Indeed, according to Bureau officials, many experienced employees retired or left the agency after the 1990 Census which affected planning efforts for the 2000 Census.

Leading organizations go beyond simply backfilling vacancies, and instead focus on strengthening both current and future organizational capacity. In this regard the Bureau acknowledges that re-engineering the 2010 Census requires new skills in project, contract, and financial management; advanced programming and technology; as well as other areas. To help address this important human capital issue, the Bureau has implemented various succession planning and management efforts to better position the agency to meet its future skill requirements.

Still, we found that the Bureau could take additional steps to enhance its succession planning and management efforts and recommended that the Bureau (1) strengthen the monitoring of its mission-critical workforce, (2) seek appropriate opportunities to coordinate and share core succession training and development programs with other outside agencies, and (3) evaluate core succession training and development programs to gauge the extent to which they contribute to enhancing organizational capacity. The Bureau agreed with our recommendations and indicated it was taking steps to implement them.

Hurricanes Katrina and Rita Highlight the Importance of Disaster Preparedness

On August 29, 2005, Hurricane Katrina devastated the coastal communities of Louisiana, Mississippi, Texas, and Alabama. A few weeks later, Hurricane Rita plowed through the border area of Texas and Louisiana.

Damage was widespread. In the wake of Katrina, for example, the Red Cross estimated that nearly 525,000 people were displaced. Their homes were declared uninhabitable, and streets, bridges, and other landmarks were destroyed. Approximately 90,000 square miles were affected overall and, as shown in figure 3, entire communities were obliterated.

The destruction and chaos caused by the storms underscore the nation’s vulnerability to all types of hazards, and highlights how important it is for government agencies to consider disaster preparedness and continuity of operations as part of their planning. We have had a preliminary discussion with the Bureau on this topic and will continue to assess the Bureau’s contingency planning as part of our oversight of the 2010 Census.

Moreover, it will be important for the Bureau to assess the impact the storms might have on its census-taking activities, as well as whether the affected areas have any special needs for data. Securing a complete count, a difficult task under normal circumstances, could face additional hurdles along the Gulf Coast, in large part because the baseline the Bureau will be working with—streets, housing stock, and the population itself—will be in flux for some time to come. According to the Bureau, different parts of the agency work on hurricane-related issues at different times, but no formal body has been created to deal with the hurricanes’ impact on the 2010 Census. The Bureau anticipates that by 2008, as it is preparing to conduct address canvassing, people will have decided whether or not to return. At that time, the Bureau believes it will be in a better position to identify vacant, occupied, and new construction for 2010.

Although Census Day is still several years away, preliminary activities, such as operations for building the Master Address File, are to occur sooner. Consequently, a key question is whether the Bureau’s existing operations are adequate for capturing the migration that has taken place along the Gulf Coast, the various types of dwellings in which people live, and the changes to roads and other geographic features that have occurred, or does the Bureau need to develop enhanced and/or additional procedures to account for them? For example, new housing and street construction could require more frequent updates of the Bureau’s address file and maps, while local governments’ participation in LUCA might be affected because of the loss of key personnel, information systems, or records needed to verify the Bureau’s address lists and maps.

It will also be important for the Bureau to work with Congress and state and local governments to determine whether the hurricane-affected areas have any special data needs to track the economic and social well-being of the region and benchmark the recovery process. Although the decennial census would not be the instrument to collect this information, it might be feasible doing so through one of the Bureau’s other survey programs. To date, the Bureau plans to do a special tabulation of its American Community Survey (ACS) data for the areas affected by Katrina that will provide information on the population that remained in the region. However, because of several methodological issues, it will not be an “official” ACS data product. The Bureau is also trying to use data from administrative records to update its population estimates of the area.

Building on these efforts, some key considerations for the future include the following: 1. How have the hurricanes affected the counties and parishes in the Gulf Coast region and what are the implications, if any, for the Bureau’s future operations? 2. Which external and internal stakeholders including federal, state, and local government agencies, as well as nonprofit organizations and specific areas of expertise need to be included in the Bureau’s decision-making process? 3. To what extent does the Bureau have a plan (including objectives, tasks, milestones, etc.) for assessing and acting on any new requirements imposed by the hurricanes? 4. Do the hurricane-affected areas have any special data requirements, and if so, how should they be addressed and which stakeholders need to be involved?

In summary, over the last few years, the Bureau has put forth a tremendous effort to help ensure the success of the 2010 Census. The Bureau is moving forward along a number of fronts, and has been responsive to the recommendations we made in our past work aimed at improving its planning process, address file, MCDs, training, human capital, and other census-taking activities. Still, some aspects of the census are proving to be problematic and a number of operational questions need to be resolved.

To be sure, challenges are to be expected in an endeavor as vast and complex as the decennial census. Moreover, shortcomings with prior censuses call for the Bureau to consider bold initiatives for 2010 that entail some risk. Thus, in looking toward the future, as the planning and testing phases of the 2010 Census begin to wind down, it will be important for Congress to monitor the Bureau’s progress in (1) identifying and diagnosing problems, (2) devising cost-effective solutions, and (3) integrating refinements and fixes in time to be evaluated during the Dress Rehearsal in 2008. Indeed, while the ramp-up to 2010 is making progress, past experience has shown that Congress has every reason to remain vigilant. As we have done throughout the past several decades, we look forward to supporting the subcommittee in its decision-making and oversight efforts.

Mr. Chairman, Mr. Clay, this concludes my prepared statement. I would be pleased to respond to any questions that you or other members of the subcommittee might have.

Contact and Acknowledgments

For further information regarding this testimony, please contact Brenda S. Farrell on (202) 512-3604, or by e-mail at farrellb@gao.gov.

Individuals making contributions to this testimony included Betty Clark, Robert Goldenkoff, Carlos E. Hazera, Shirley Hwang, Andrea Levine, Anne McDonough-Hughes, Lisa Pearson, Michael Volpe, and Timothy Wexler.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.