2000 Census: Preparations for Dress Rehearsal Leave Many Unanswered Questions
Published: 1998-03-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the status of key activities that the Bureau of the Census plans to test as part of its dress rehearsal for the 2000 Census.
section_title: What GAO Found
paragraphs: GAO noted that: (1) address list development, local outreach and promotion, staffing, and statistical sampling are all still facing the developmental and implementation changes that led GAO in 1997 to raise concerns about the high risk of a failed census in 2000; (2) this situation has arisen in part because Congress and the administration have yet to reach agreement on key aspects of the census design; (3) an accurate address list and associated maps are essential for ensuring that households receive census questionnaires and for establishing the universe of housing units for sampling and statistical estimation procedures; (4) accurate maps are critical for locating households for congressional reapportionment and redistricting; (5) the Bureau's initial plan for creating the address list and maps for the 2000 Census called for: (a) combining addresses provided by the Postal Service with the Bureau's 1990 Census address file and mapping database; and (b) submitting these addresses and maps to local governments for their review; (6) the Bureau has since concluded that the resulting address file might not meet its goal of being 99 percent complete; (7) in September 1997, the Bureau announced plans to reengineer its process for creating the address file for the 2000 census, in part by hiring temporary staff to physically verify the accuracy of all addresses by canvassing streets; (8) this reengineered approach has not been tested because of a lack of resources, and will not be tested because the Bureau does not have sufficient time to do so; (9) as part of its reengineering effort, the Bureau plans to seek earlier local government review of addresses from the Postal Service and the Bureau's 1990 Census file; (10) the Bureau's approach still will not address some of the problems encountered by local officials in their reviews; (11) to boost the questionnaire mail response rate and thus reduce its costly nonresponse follow-up workload, the Bureau plans to partner with local governments, schools and other community organizations to promote the census and has initiated a paid advertising campaign that is being developed by a private contractor; (12) the Bureau estimates that it will need to recruit over 2.6 million applicants to fill about 295,000 office and field positions; (13) the dress rehearsal does not and cannot provide an adequate test of the effectiveness of the Bureau's staffing strategy for the 2000 Census; and (14) several missteps by the Bureau in drawing a sample for the Integrated Coverage Measurement have raised GAO's concerns about the Bureau's quality control procedures.

section_title: Letter
paragraphs: This report, which was prepared at your request, reviews the status of key activities that the U.S. Bureau of the Census plans to test as part of its dress rehearsal for the 2000 Census. To the extent that it mirrors the decennial, the dress rehearsal could foreshadow how well these activities might work in 2000, and thus indicate where additional congressional and Bureau attention is needed now to ensure a successful census.
As agreed with the Committee, this report reviews the results to date of the Bureau’s efforts to create a complete and accurate address list; increase the mail response rate through outreach and promotion; staff its operations with an adequate workforce; and reduce costs and improve accuracy through sampling and statistical estimation methods.
We are sending copies of this letter to the Chairman and Ranking Minority Member, Subcommittee on the Census, House Committee on Government Reform and Oversight; other appropriate congressional committees; the Secretary of Commerce; the Director, Bureau of the Census; and other interested parties. Copies will be made available to others on request.
Please call me or J. Christopher Mihm, Associate Director, on (202) 512-8676 if you have any questions concerning this report. The major contributors to this report are listed in appendix II.
subsections: section_title: Introduction
paragraphs: The decennial census is the nation’s most comprehensive and expensive statistical data-gathering program. An accurate population count is critical because, as required by the Constitution, census results are used for allocating seats in the House of Representatives. Census data are also used for redistricting within states, distributing billions of dollars in federal funding, guiding investment decisions of the public and private sectors for the ensuing decade, and providing a baseline for countless other statistical measurement programs.
Unfortunately, as we noted in a previous report, the nation runs the risk of a very expensive and seriously flawed census in 2000. To assess the Bureau of the Census’ progress in minimizing risks, we reviewed the status of key activities that the Bureau plans to test as part of its dress rehearsal for the next decennial census: address list development, local outreach and promotion, staffing, and statistical sampling. To the extent that the dress rehearsal accurately mirrors the actual census, the results of the dress rehearsal could indicate what might occur in 2000.
subsections: section_title: The Census Dress Rehearsal Is to Provide a Full-Scale Demonstration of Most Procedures Planned for 2000
paragraphs: According to the Census Bureau, the dress rehearsal for the 2000 Census is designed to test the various operations, procedures, and questions that are planned for the decennial under as near census-like conditions as possible. In July 1996, the Bureau announced that Sacramento, CA; 11 counties in the Columbia, SC, area; and Menominee County, WI, including the Menominee American Indian Reservation, had been selected as sites for the 2000 Census dress rehearsal.
According to the Bureau, Sacramento was selected because its diverse population will allow the Bureau to test estimation methods designed to reduce differential undercounts. Sacramento is also a primary media market, thus enabling the Bureau to test its paid-advertising campaign. The Columbia site was selected, among other reasons, for its mixed rural and urban features and because its relatively high proportion of African-Americans will give the Bureau an opportunity to test methods of reducing the differential undercount for this population group. The Menominee site was included to allow the Bureau to test measures aimed at reducing the undercount of American Indians.
Administratively, the Bureau’s Seattle Regional Office is responsible for conducting the Sacramento dress rehearsal, while its Charlotte, NC, and Chicago, IL, Regional Offices, respectively, are responsible for the South Carolina and Menominee dress rehearsals.
An important aspect of the dress rehearsal activities is Dress Rehearsal Census Day, which is scheduled for April 18, 1998. However, as is the case with the actual census, the Bureau’s dress rehearsal activities span a much wider period of time than 1 day. Following the selection of the dress rehearsal sites in July 1996, the Bureau developed preliminary mailing lists and materials for these locations, contacted local governments at the three sites, and conducted staffing activities to hire temporary census employees in those locations. Similarly, after April 18, the Bureau is to develop its census count by conducting the necessary follow-up activities at nonresponding households and completing other fieldwork.
section_title: A Demonstration of the Bureau’s Plans Is Critical Given Past Problems
paragraphs: A well-planned and -executed dress rehearsal should provide the Bureau with an opportunity to demonstrate the feasibility of its design for the 2000 Census. This demonstration is particularly important considering the problems the Bureau has experienced to date and the controversy that surrounds its design.
Since 1970, when measured in constant dollars, census costs have increased faster than inflation, even after allowing for population growth. In 1990 constant dollars, total census cycle costs were $0.7 billion in 1970, $1.8 billion in 1980, and $2.6 billion in 1990. The Bureau expects that the escalating cost of conducting the census will continue; in March 1997, the Bureau estimated that the 2000 Census will cost at least $4.0 billion in 2000 dollars.
Unfortunately, the nation’s growing investment in the census has not generated more accurate results. Although the 1990 Census was the most expensive in history, the results were less accurate than those of the 1980 Census. The Bureau estimated that about 6 million persons were counted twice in the 1990 Census, while 10 million persons were missed. These errors were particularly worrisome because the Bureau’s evaluation showed a persistent differential undercount of minority groups and renters, particularly those living in rural areas.
The cost of a census is largely influenced by the relative need for the Bureau to follow up on nonresponding housing units. Since the 1970 Census, the Bureau has used a “mail-out/mail-back” approach for counting the vast majority of the population whereby the Bureau develops an address list of the nation’s housing units and mails a questionnaire to each one. Although households are to mail back completed forms, many millions do not. This nonresponse requires the Bureau to hire hundreds of thousands of temporary employees, called enumerators, to follow up on each nonresponding housing unit by visiting each unit and gathering census information. The mail response rate to the census questionnaire has dropped from 78 percent in 1970 to 65 percent in 1990, thus increasing the Bureau’s costly nonresponse follow-up workload.
section_title: Increasing Congressional Concern Has Led to Changes to the Dress Rehearsal
paragraphs: The Bureau’s plans for the 2000 Census include the use of sampling and statistical estimation. Initially, the Bureau planned to use these methods at the three dress rehearsal sites just as it plans to do nationwide in 2000. Congressional concern over the validity and legality of the Bureau’s sampling and statistical estimation methods have led to recent changes to the dress rehearsal. According to Department of Commerce officials, as part of a compromise between the administration and Congress over the Bureau’s fiscal year 1998 appropriation, it was agreed that the Bureau would use sampling and statistical estimation methods only in the Sacramento site for nonresponse follow-up and to improve the accuracy of the population count. In the Columbia site, the Bureau is to hire enumerators to follow up on all nonresponding households, just as it did nationally in the 1990 Census. At the Menominee dress rehearsal site, the Bureau is to do a 100 percent nonresponse follow-up but is to use sampling to improve the accuracy of the population count. Although use of the different methods at the dress rehearsal sites invites a comparison of the results, geographic, demographic, and possibly other differences between the dress rehearsal locations preclude such a comparison.
The Bureau’s fiscal year 1998 appropriation act also allows for expedited judicial review of sampling and created an eight-member Census Monitoring Board that is to oversee all aspects of the preparation and implementation of the 2000 Census and report its findings to Congress.
section_title: Objectives, Scope, and Methodology
paragraphs: Concerned about the risk of a failed census, the Chairman and Ranking Minority Member, Senate Committee on Governmental Affairs, asked us to review the status of key census-taking activities that the Bureau is to test during the 1998 dress rehearsal. In response, we examined the Bureau’s efforts to (1) create a complete and accurate address list, (2) increase the mail response rate through local outreach and promotion, (3) staff census-taking operations with an adequate workforce, and (4) implement its sampling and statistical estimation methodology. If designed and implemented as much like the actual census as possible, the dress rehearsal could yield results indicative of what might occur in 2000.
To examine the status of address list development, local outreach and promotion, and hiring, we obtained documents that describe the Bureau’s plans, procedures, and progress for each activity and interviewed Bureau headquarters officials and, where applicable, regional office officials. We also reviewed the Bureau’s analysis of missed or duplicated housing units in the 1990 census; the Bureau’s analyses of missed or duplicate housing units in its 1995 Census Test conducted in Paterson, NJ, and Oakland, CA; and its analysis of the completeness of address lists for seven locations around the nation. This was done to review the accuracy and completeness of the Bureau’s address list.
To help verify the Bureau’s information and also obtain a local perspective on the dress rehearsal activities, we (1) made site visits to the Sacramento and South Carolina dress rehearsal jurisdictions; (2) conducted in-person and telephone interviews with local officials on their experiences in reviewing address lists, promoting the census, and recruiting and hiring census workers; and (3) where applicable, obtained relevant documents on these activities. Menominee officials were contacted by telephone. To obtain an indication of the task the advertising contractor faces in developing an effective promotional campaign, we observed focus group sessions the contractor scheduled with different population groups as part of its market research at the dress rehearsal sites.
To examine the Bureau’s efforts to implement its sampling and statistical estimation methodology, we reviewed Bureau documents related to sampling and estimation, interviewed Bureau officials who are responsible for drawing the sample for the dress rehearsal, and reviewed the Bureau’s methodology for drawing the sample.
We conducted our audit work at the South Carolina and Sacramento dress rehearsal sites; the Bureau’s Charlotte and Seattle regional offices; as well as in Washington, D.C., and Dallas, TX, between September 1997 and February 1998 in accordance with generally accepted government auditing standards.
We requested comments on a draft of this report from the Secretary of Commerce or his designee. On March 18, 1998, the Acting Director, Bureau of the Census, provided us with comments on the draft report. These comments are presented and evaluated in chapter 6 and reprinted in appendix I.
section_title: The Quality of the Address List Has Significant Implications for the 2000 Census
paragraphs: A quality address list—one that is complete and accurate—and precise maps are the fundamental building blocks of a successful census. Accurate addresses are essential for delivering questionnaires, avoiding unnecessary and expensive follow-up efforts at vacant or nonexistent residences, and establishing a universe of households for sampling and statistical estimation. Accurate maps are critical for counting the correct portions of the population in their correct locations—the cornerstone of congressional reapportionment and redistricting.
For the 2000 Census, the Bureau estimates that it needs to identify and locate 118 million housing units and to create a database of addresses, known as the Master Address File (MAF). To build the MAF, the Bureau originally planned, in part, to (1) use addresses provided by the Postal Service, (2) merge these addresses with the address file the Bureau created during the 1990 Census, (3) conduct limited checks of the accuracy of selected addresses, and (4) send the addresses to local governments for verification. However, the Bureau’s experience during the dress rehearsal shows that its procedures for developing an accurate MAF and maps have weaknesses that, despite the Bureau’s recent improvement efforts, may not necessarily result in a MAF or maps of sufficient accuracy to ensure all households receive a census questionnaire. Because of these procedural weaknesses, the Bureau is “reengineering” how it builds the MAF by physically verifying all addresses and by encouraging earlier local government reviews of address lists and maps created from the Postal Service and the Bureau’s 1990 address file. Although this reengineered approach may address several problems, the Bureau will not be testing this new approach before the 2000 Census.
subsections: section_title: The Postal Service’s Address File May Not Be Sufficiently Accurate or Complete to Meet the Bureau’s Needs
paragraphs: To reduce cost and improve the accuracy of the 2000 Census, the Bureau, in developing the MAF for city-style addresses for the census, originally planned to combine the Postal Service’s Delivery Sequence File (DSF), which includes all mail delivery points recognized by the Postal Service, with the address file created during the 1990 Census. The Bureau also planned to conduct a limited amount of canvassing in each city to ensure that certain areas, such as those that have experienced growth, had accurate and complete addresses or those that have multiunit (e.g., apartment) buildings had consistent counts of apartment units. In areas with mostly non-city-style mailing addresses, such as rural areas that have rural delivery or post office mailing addresses, the Bureau plans to create a MAF as it did for the 1990 Census by having census enumerators visit all housing units in the designated areas to identify their physical locations, and to obtain other data such as each occupant’s name, mailing address, and telephone number.
On the basis of its analyses of the DSF, the Bureau has concluded that the DSF, combined with targeted canvassing in city-style address areas, cannot be relied upon to achieve its goal of having a MAF for the 2000 Census that has no more than 1 percent net housing unit undercoverage and no more than 1 percent duplicate housing coverage. One of the major problems associated with having an inaccurate address list is the cost incurred in following up on questionnaires that are not mailed back. In the 1990 Census, we reported that the cost to follow up on questionnaires accounted for 57 percent, or $730 million, of the $1.3 billion spent on data collection. A portion of that amount was caused by inaccuracies in the address list. For example, according to the Bureau, the cost of unnecessary follow-up work at housing units that were either vacant or nonexistent was about $317 million in 1990, a total that included the direct cost of hiring enumerators and providing training and travel expenses, as well as the indirect cost of office space and equipment, recruiting, and administration. To estimate the potential quality of the MAF for the 2000 Census, the Bureau has analyzed address lists created for a few selected locations around the nation. Following the 1995 Census Test in Paterson, NJ, and Oakland, CA, the Bureau found that field-verifying addresses (e.g., precanvassing), local government review of the MAF, and a final Postal Service check added 6.4 percent and 3.6 percent of the total addresses on the MAF for those locations, respectively. Precanvassing and local government review of the MAF also deleted 9 percent and 6 percent of the addresses on the MAF for Paterson and Oakland, respectively. The Bureau also examined the quality of address files created for the American Community Survey test sites in seven locations. The Bureau’s estimates of missed housing units ranged from a low of 0.0 percent in Rockland County, NY, to a high of 2.4 percent in Harris County, TX. The Bureau’s estimates of duplicate or nonexistent housing units ranged from a low of 0.0 percent in Multnomah County, OR, to a high of 11.7 percent in Rockland County.
According to the Bureau, incidents of missed housing units occur because the DSF is not (1) completely up-to-date in terms of new housing construction; and (2) uniformly current because the DSF reflects address changes in certain areas faster than in other areas. Missed housing units can also occur because the Postal Service may define the delivery points for mail differently than the Bureau defines separate housing units. This could occur, for example, when the Postal Service uses one delivery point for all mail delivered to a multiunit apartment building. Further, erroneous addresses can be introduced into the MAF when local governments provide the Bureau with addresses generated from locally developed addressing systems (e.g., emergency services) that may differ from the addresses contained in the DSF and used for mail delivery.
According to Bureau officials, in June 1997, the Bureau initiated an effort to determine whether it could improve the process for building the MAF for the 2000 Census, and, if so, how. As part of that effort, the Bureau assembled a team that included tribal, state, and local government representatives to obtain outside reaction to the Bureau’s original plan to develop its address list by relying primarily on the DSF and targeted canvassing, and to discuss new approaches to developing the address list. The team developed a modified, “reengineered” plan for address list development that includes, for areas that contain predominantly city-style addresses, verifying all addresses (100 percent block canvassing) through field visits, as was done during the 1990 Census. The Bureau estimates that the reengineered plan will add $108.7 million to its address list development procedures, which are already budgeted at $357.2 million.
According to the Bureau, the reengineered plan will allow it to achieve its goal of a MAF that has no more than 1 percent net housing unit undercoverage and no more than 1 percent duplicate housing coverage. The Bureau’s initial plan for developing the MAF for city-style addresses included combining the DSF and the 1990 Census Address Control File (ACF); conducting limited, targeted canvassing; and local government review. Its decision to reengineer the MAF building process was based on Bureau officials’ judgment, following their review of existing data, that the initial plan would not result in the high coverage rate desired. It is not known whether the reengineered plan will result in the high accuracy rate the Bureau wants to attain because the Bureau has not tested this plan to determine the number and nature of missed or erroneously included housing units that would occur by using this process. For city-style addresses in the Bureau’s 1995 Census Test, the Bureau combined the DSF, the ACF, 100-percent canvassing, local address review, and a Postal Service address accuracy check, to compile the MAF. However, Bureau officials stated that they lacked the resources to determine the final housing unit undercount and the duplicate housing unit rates in the 1995 Census Test MAF as was done following the 1990 Census.
section_title: Local Reviews May Be Too Sporadic and Contain Too Many Obstacles to Consistently Confirm or Improve the MAF’s Accuracy
paragraphs: Local address review has not progressed smoothly at the dress rehearsal sites. Many governments did not participate in local review, while some governments that did participate cited time and resource constraints and limited assistance from the Bureau as impediments to their reviews.
Under the Bureau’s Local Update of Census Addresses (LUCA) program, local governments and Indian tribes owning lands are to be given an opportunity to verify address list information. In 1990, local governments were given the opportunity to review Bureau counts of housing units, as well as boundary maps to identify any major discrepancies. The 1994 Census Address List Improvement Act extended the Bureau’s confidentiality provisions to local governments, and beginning with the 2000 Census, the Bureau plans to provide local government officials with detailed address information. The importance of LUCA in improving the quality of the Bureau’s MAF was demonstrated during the 1995 Census Test. During that test, the Bureau found that LUCA alone added, deleted, corrected, or moved 7.1 percent of the addresses in Paterson and 1.6 percent of the addresses in Oakland.
subsections: section_title: Time and Resource Constraints Limited Local Government Participation
paragraphs: The dress rehearsal locations include 62 jurisdictions—Sacramento, Menominee County, and 60 jurisdictions (11 counties and 49 towns and cities) in the 11-county South Carolina area. Sacramento and Columbia as well as Menominee County and the state of South Carolina all were involved with the LUCA program, although participation was far from universal among the South Carolina jurisdictions. In South Carolina, the Bureau’s Charlotte Regional Office officials (1) contacted the mayors of the 49 municipalities and the chairs of the 11 county councils, (2) made presentations to county councils and the City Manager for Columbia about the importance of reviewing the census address list, and (3) mailed numerous letters to the jurisdictions. However, despite the Bureau’s efforts, it received responses to the LUCA review, in terms of annotated maps and/or address changes, for only 34 of the 60 jurisdictions. South Carolina’s Office of Research and Statistics, which maintains geodetic data for accurate large-scale mapping, provided assistance to or conducted LUCA for an additional 10 of 60 the jurisdictions for a total participation rate of 44 jurisdictions. Due to resource constraints, this office was unable to assist two other jurisdictions.
According to Bureau regional officials, reasons jurisdictions gave for not participating in the LUCA review included lack of resources and/or lack of information to update address lists or maps. Our discussions with officials from 5 of the 16 South Carolina area jurisdictions that did not participate in LUCA and that responded to our inquiries, confirmed that the jurisdictions’ primary reason for not participating was a lack of resources. For example, an official from one South Carolina town said that the town would have liked to participate, but that there was no one locally who would agree to work on or help out with LUCA. Similarly, an official from another South Carolina town said that it too would have liked to participate. However, she noted that she would have been tasked to do the job, and that there was insufficient time to perform her full-time job with the town as well as review the Bureau’s address list and maps.
Our discussions with officials from 12 South Carolina area jurisdictions that did participate in LUCA and that responded to our inquiries, indicated that the potential cost of participating in LUCA can be high. For example, an official in one South Carolina county estimated the cost to be $5,000, with staff having to perform LUCA as well as their normally assigned duties in the tax assessor’s office. An official from another city was unable to provide a cost estimate, but indicated that the city’s Geographic Information System specialist had spent about 4 weeks full-time on the effort, and that several others had spent part of their time on the effort. An official in another city said that the city committed 4-1/2 weeks to LUCA. This official said that, during that time, two people worked full-time, 9 hours a day, driving down each street in the city. After that initial canvassing, two people worked full-time and two other people worked part-time for the remaining 2 weeks correcting the maps and address lists.
Sacramento officials estimated that city officials spent about 90 hours over the course of 5 weeks, while the officials from the local area council of governments spent about 450 hours. Even with these resources, these officials estimated that only about half of the amount of work required for LUCA was accomplished. This work included reconciling addresses in blocks where the city and the area council of governments believed the Bureau missed addresses and reviewing and correcting maps.
Even with the resources committed to conducting reviews in those jurisdictions that participated in LUCA, officials in the six jurisdictions we contacted and that responded to our inquiries said that the amount of time allowed for LUCA during the dress rehearsal—about 5 weeks—was insufficient. According to Sacramento officials, time constraints resulted in the city only being able to completely review 17, and partially review another 15, of the 48 maps the Bureau provided the city; 16 maps were not reviewed at all. Officials from one South Carolina jurisdiction said that because of insufficient time, local officials did not focus any efforts on reviewing addresses for possible deletions. Instead, they concentrated their efforts on adding addresses even though they acknowledged that these addresses may already have been located elsewhere in the address files without their knowledge.
section_title: Limited Bureau Assistance and Poor MAF and Map Quality Hampered Local Review
paragraphs: Those jurisdictions participating in LUCA reported several obstacles that made their reviews more difficult. For example, officials in some jurisdictions stated that the Bureau was able to provide little assistance to them while they were reviewing the address lists and maps. Furthermore, the accuracy, completeness, and format of the address lists and maps hindered their review.
According to Bureau officials, although the Bureau’s Seattle and Charlotte Regional Offices each designated a LUCA coordinator, the Seattle coordinator for Sacramento was responsible for just a single jurisdiction. In contrast, the Charlotte LUCA coordinator was responsible for all 60 jurisdictions in the South Carolina dress rehearsal site. This difference in workload could be responsible for the different levels of satisfaction between the two sites with the quality of the Bureau’s assistance, and suggests that the Bureau’s workload in 2000 may significantly limit its ability to provide needed assistance to localities.
For example, according to Sacramento officials, the Bureau’s Seattle Regional Office staff was available at all times during the LUCA review, and the level and quality of Bureau support was sufficient. However, a South Carolina county official said that, while early contact with the Bureau was open and forthcoming, during the LUCA review Bureau staff provided little to no help. According to this official, when the county contacted the Bureau about problems it was experiencing with the Bureau’s electronic files and maps, it was told that the Bureau was aware of these problems but that it could not provide any guidance or help. Similarly, officials from one of the South Carolina cities said that the Bureau was of limited help in providing guidance and assistance during their review of the files and maps. According to these officials, they were told that because of the Bureau’s limited resources, local governments were relatively “on their own” when conducting LUCA.
The Bureau’s assistance to local governments in 2000 is expected to be significantly less than that provided to South Carolina during the dress rehearsal. According to a Charlotte Regional Office official, while the Bureau had one staff person available to assist the 60 local jurisdictions during the dress rehearsal, this level of support will greatly diminish for the 2000 Census. According to Bureau officials, on average, each region will have about 11 partnership specialists for the 2000 Census, of which about half are to focus primarily on LUCA. Therefore, the LUCA coordinators in the Charlotte office would be responsible for about 2,200 communities in that 5-state region, or about 200 to 400 communities each.
The accuracy, completeness, and format of the address lists and maps were also an issue to participating jurisdictions, and an issue that could reduce local governments’ ability to improve the accuracy of the MAF and maps in 2000. According to Sacramento officials, the Bureau’s address file for the city contained about 198,000 addresses, which were more addresses than were contained in Sacramento’s database of addresses. Sacramento officials found that the Bureau’s address file included addresses outside the city limits, addresses not assigned to any block or tract number, and commercial addresses. This significantly contributed to problems in reviewing the Bureau’s address file. Sacramento officials also said that the electronic address file provided by the Bureau did not include information on where each address was located on the maps, which created additional work for the city.
Similar concerns were voiced by South Carolina officials. These officials said that the maps did not include street corrections, street additions, and block adjustments made to the Bureau maps about 1-1/2 years earlier as part of the Bureau’s Block Boundary Suggestion Program. Furthermore, the maps did not include changes in city boundaries due to annexations that are provided annually to the Bureau under the Bureau’s Boundary and Annexation Survey. As a result, according to officials from the city of Sacramento and the Sacramento Area Council of Governments, they expended considerable time reviewing the LUCA maps to make corrections that they had provided earlier to the Bureau. Similar problems were voiced by South Carolina officials. For example, according to officials in one South Carolina county, the same errors were on the LUCA maps that the county has been trying to correct since the 1990 Census.
According to Bureau officials, maps may contain errors for a variety of reasons, including the Bureau’s inability to make corrections on some Bureau maps when local governments provide corrections and revisions on their own maps, as opposed to the Bureau’s maps. The Bureau cannot make some of the local government’s corrections and revisions because Bureau maps are not always spatially accurate and thus Bureau staff may be unable to make the fine corrections necessary to accurately reflect changes on locally produced, spatially correct maps.
section_title: The Bureau’s Reengineering Plans Address Some, but Not All, Concerns Cited by Local Officials About LUCA
paragraphs: According to the Bureau, the reengineered approach to developing its address list addresses several problems cited by jurisdictions that participated in LUCA during the dress rehearsal. However, while the revised approach does address some problems, it does not fully address them all. Furthermore, the feasibility of the Bureau’s plan remains questionable.
To remedy the problem of lack of sufficient time to perform LUCA, the Bureau plans to begin the LUCA process earlier and allow more time for local review. Rather than have LUCA begin in late 1998 and continue until the second quarter of 1999, the Bureau plans to begin LUCA in early 1998 and continue it through the second quarter of 1999.
To address the lack of participation in LUCA, the Bureau plans to establish stronger relationships with state data centers, metropolitan and regional planning organizations, and councils of government, especially in areas where local governments are unable to participate. However, based on the Bureau’s dress rehearsal experience, it is questionable whether such organizations can provide the assistance envisioned by the Bureau. For example, according to officials from the South Carolina Office of Research and Statistics, the organization lacked sufficient resources to provide assistance to all of the jurisdictions in the SC dress rehearsal site that requested help. Should assistance be required on a statewide basis, these officials said that the research and statistics office would not have sufficient resources to provide it.
The Bureau’s reengineered approach does not address the quality of the address lists or maps cited by officials from some of the dress rehearsal jurisdictions. Although Bureau canvassing could improve the quality of the address lists and maps to a certain extent, because the Bureau’s reengineered approach provides for an earlier LUCA, canvassing will not occur in areas having city-style addresses until after local governments have completed their review of addresses and maps. The Bureau’s reengineered approach also does not address the level of Bureau assistance available to local governments that choose to participate in LUCA, even though it does call for providing more training. Further, given the Bureau’s decision to reengineer its address list development efforts, the Bureau will not be able to test its revised approach before implementing it for the 2000 Census.
section_title: The Bureau’s Mail Response Rate Objective Could Be Difficult to Achieve Because of Obstacles to Implementing Its Outreach and Promotion Strategy
paragraphs: To increase awareness of the 2000 Census and boost mail response rates, the Bureau plans an extensive outreach and promotion campaign. Effective outreach and promotion are essential because they have the potential to boost the mail response rate, thereby reducing the Bureau’s nonresponse follow-up workload and, thus, the cost of the census. Through targeting, outreach and promotion can also help reduce any undercount of traditionally hard-to-enumerate populations.
The Bureau expects that its outreach and promotion efforts, combined with other initiatives, such as simplified census questionnaires, should produce a mail response rate of 66.9 percent for the 2000 Census. This is 12 percentage points higher than the 55-percent response rate that the Bureau expects it would achieve without these activities and slightly higher than the 65-percent response rate achieved in the 1990 Census. Nevertheless, the Bureau’s experience thus far during the dress rehearsal suggests that, in 2000, this goal might be difficult to achieve.
subsections: section_title: The Bureau Has Initiated Several Outreach and Promotion Activities
paragraphs: According to the Bureau, the success of its outreach and promotion efforts will depend heavily on the effectiveness of the partnerships it hopes to build with state, local, and tribal governments; nongovernmental organizations; business; and the media. As a matter of long-standing policy, the Bureau has maintained that it cannot seek to shoulder the major financial burden of local census outreach and promotion efforts. Thus, the Bureau is placing a priority on working with partners because they can, among other things, help spread the word about the census, encourage active participation, dispel myths, and issue public statements of endorsement. Among the major activities we examined for this report were the Bureau’s efforts to form Complete Count Committees (CCC), implement its Census in Schools Project, and contract-out for a paid-advertising campaign.
One of the primary goals of the CCCs is to heighten public awareness of the census through community outreach activities. Bureau guidance recommends that local governments form CCC’s consisting of elected officials, business leaders, minority groups, religious organizations, community-based organizations, and other individuals and groups. Such partnerships could provide the Bureau with local knowledge, experience, and expertise to help it take a more accurate census and ensure productive use of the Bureau’s limited resources. For example, CCCs could assist the Bureau by alerting Bureau officials to the existence of hard-to-count population groups within their communities. These groups could then be targeted for more intensive and/or specific types of advertising.
Bureau guidance provides potential CCCs with specific projects that they could undertake to reach individuals in their communities. Such projects could include working with businesses to (1) obtain their commitment to display promotional flyers about the census, (2) insert reminders about the census in customers’ billing statements or employees’ paychecks, or (3) set up a census assistance table where volunteers would assist people in completing their census forms. Other projects could involve obtaining commitments from local radio stations or newspapers to provide media coverage and publicity. The Sacramento and Menominee County CCCs, as well as CCCs in some of the South Carolina counties, have initiated these and/or similar activities.
Through its second partnership activity, the Census in Schools Project, the Bureau hopes to increase response rates by using students to encourage and, where necessary, help their parents or guardians complete census forms. The schools project is also intended to (1) organize millions of teachers, students, and families to support the census: (2) build a database of interested teachers who may be recruited as census workers; and (3) improve students’ understanding of the census process and why it is important to the country. As currently planned, the schools project is to include grades kindergarten through 12 as well as classes for adult English as a second language, citizenship, and high school equivalency degrees. The project also is to focus on five types of schools: public, private, parochial, Bureau of Indian Affairs, and Department of Defense. College and university students will be not be included in the schools project because the Bureau believes that these students can be better reached through traditional advertising.
A third partnership activity is the Bureau’s paid-advertising campaign. According to the Bureau, the campaign is intended to increase mail returns from the general public, targeted audiences, and historically undercounted populations. In the 1990 Census, the Bureau relied on pro bono public service advertising to get its message across. However, such advertising had its limitations. For example, the Bureau has noted that television announcements were often aired at non-prime-time hours when audiences were relatively small. For the 2000 Census, the Bureau concluded that a paid-advertising campaign would allow it to better choose where and when to place advertisements, more effectively reach hard-to-count populations, and present a more uniform message.
On October 10, 1997, the Bureau announced that it had awarded its 2000 Census paid-advertising contract to Young & Rubicam (Y&R), which is a private advertising agency. A Y&R subcontractor, Scholastic, Inc., is to develop a package of educational materials and a marketing plan for the Bureau’s schools project.
According to Bureau officials, the Bureau has budgeted about $230 million for 2000 Census outreach and promotion activities. Of that amount, about $100 million is for the Y&R paid-advertising campaign. Of the $100 million, about 80 percent has been earmarked for buying advertising in print and the broadcast media. The remaining 20 percent is to be used for such activities as contract administration and overhead. Of that 20 percent, a portion is to be used for an independent evaluation of the effectiveness of the paid-advertising campaign.
Another $100 million of the $230 million outreach and promotion budget is to be used to hire as many as 320 full-time equivalent (FTE) partnership specialists, including government, community, and media specialists. The remaining $30 million is to fund (1) per diem and travel expenses for members of census advisory committees, (2) the Census in Schools Project, and (3) certain local promotional activities, such as printing.
section_title: Effectively Implementing the Bureau’s Outreach and Promotion Activities Could Be Problematic
paragraphs: Although the Bureau’s plan to publicize the 2000 Census was designed to address the outreach and promotion problems encountered during the 1990 Census, the results to date of the dress rehearsal suggest that several obstacles may diminish its effectiveness. These obstacles include inconsistent levels of participation by local communities, a lack of resources at both the Bureau and local level, and resistance to the Bureau’s message that the census is important. As a result, the Bureau may have difficulty achieving its goal of a 66.9 percent mail response rate in 2000.
subsections: section_title: Participation in Outreach and Promotion Activities at the Dress Rehearsal Sites Has Been Inconsistent
paragraphs: Although the Sacramento and Menominee County CCCs appear to be active, efforts to establish CCCs in South Carolina have had mixed results. Of the 11 counties and City of Columbia included in the South Carolina dress rehearsal, just 3 counties and Columbia had active CCCs at the time of our review. The remaining eight counties either had not formed CCCs or those that were formed had become inactive.
Reasons for the nonfunctioning CCCs in South Carolina varied and were at times beyond the Bureau’s control. For example, according to a Bureau official, one county’s CCC disbanded after a key member left. In at least two instances, however, it appears that the Bureau was at least partly responsible for the operational problems that caused the CCCs to suspend their activities. For example, one local official said that CCC members became frustrated and dissatisfied with the lack of organization and leadership from representatives of the Bureau’s regional office. An official from another CCC said that vague communications between the Bureau and the CCC played a role in the CCC’s cessation of activities. The official said the Bureau did not set clear expectations for what the CCC was supposed to do, and that the Bureau had supplied only minimal guidance and support.
Communication difficulties appeared to be a problem with several active South Carolina CCCs as well. Four of the six active CCCs we contacted at the three dress rehearsal sites indicated that the Bureau did not set clear expectations for their CCCs, especially when they were first initiated, and/or Bureau guidance and literature has been minimal.
section_title: Local and Bureau Resources Affected Results to Date of Outreach and Promotion Efforts at the Dress Rehearsal Sites
paragraphs: Local and Bureau resources may not be adequate for effective outreach and promotion. While the Bureau generally plans to provide communities with in-kind services (e.g., copies of flyers), local communities are to provide critical funding during the 2000 Census. In practice, however, this arrangement has led to some operational problems with the CCCs at the dress rehearsal locations. For example, officials from five South Carolina counties and the City of Columbia indicated that resources to carry out promotional activities were limited. In Sacramento, a city official said that much of what the city’s CCC proposed to do in the way of media advertising will depend on the funding available from the Bureau’s advertising contractor and any in-kind services provided by the Bureau.
Moreover, although local officials said that some local businesses have agreed to undertake such activities as displaying promotional flyers, as a rule, local businesses have not provided and likely will not provide funding for promoting the dress rehearsal. For example, in South Carolina, officials from five counties said that funding and in-kind services from local businesses would probably not be available. The Bureau’s partnership specialist for the South Carolina site acknowledged that local business support has been sporadic and that business partnerships the Bureau had hoped for have not yet occurred. In Sacramento, a local official noted that while businesses would likely be enthusiastic about certain in-kind services, such as displaying posters and including flyers in mailings, they probably would not provide any funding. At the Menominee County dress rehearsal site, a tribal official noted that the reservation contained few businesses, virtually all of the businesses were small, and none had the resources to contribute to promoting the census.
The costs of supporting CCC activities typically were absorbed by local governments. An official from one South Carolina county estimated that the county’s CCC has spent 100 staff hours on CCC initiatives, while an official on the City of Columbia’s CCC said its CCC members have spent about 240 staff hours on outreach and promotion activities as of early December 1997 and will probably spend an equal amount before the dress rehearsal is finished. An official from another South Carolina county estimated that its CCC has spent $2,000 in cash and in-kind services for outreach and promotion efforts and that it will spend about $5,500 more before the dress rehearsal is completed. An official from another South Carolina CCC told us that outreach and promotion activities have cost about $8,000 to date, and the official anticipated that the CCC would spend an additional $27,000 before the dress rehearsal is over.
As previously noted, the Bureau ultimately plans to fill as many as 320 FTE partnership specialist positions for the decennial census in its 12 regional offices. Given the approximately 3,100 counties and county equivalents that comprise the nation, each full-time partnership specialist’s caseload would average 10 counties. This is only slightly less than the workload of the Bureau’s South Carolina partnership specialist, who, as previously described, was responsible for mobilizing outreach and promotion efforts in 11 counties and the City of Columbia and achieved only mixed results. (The South Carolina partnership specialist’s workload did not go unnoticed; several local officials from five of the South Carolina dress rehearsal sites observed that the partnership specialist might have been stretched too thin.) Furthermore, in addition to outreach and promotion efforts during the 2000 Census, the partnership specialists will be responsible for additional activities, such as responding to media inquiries, obtaining space for recruiting and training census workers, and assisting with LUCA reviews.
Local resources for promoting the census was an issue during the 1995 Census Test. As we noted in a previous report, the 1995 Census Test indicated that cooperative ventures with local governments provided a way to promote public participation in the census. However, as we reported, local governments in urban areas where response rates were lowest reported that their lack of funding to promote the census was problematic.
The Bureau is also implementing the Census in Schools Project at the dress rehearsal sites and, as with the CCCs, is encountering inconsistent participation. In South Carolina, the Bureau’s partnership specialist told us that six counties were participating in the program at the time of our study. As was the case with the South Carolina CCCs, county officials involved with the schools project cited problems such as unclear expectations from the Bureau and/or lack of resources as the reasons for their difficulties.
section_title: The Bureau’s Paid-Advertising Campaign Faces Challenges
paragraphs: The Bureau’s national advertising contractor, Y&R, began its efforts for the dress rehearsal in mid-December 1997 by convening a series of focus group interviews with different segments of the population at the dress rehearsal sites. According to Bureau officials, the purpose of these focus groups was to build on earlier, more extensive research that was conducted by the Bureau on personal reactions to the census, and to test proposed advertising themes and approaches that could be used to reach specific, targeted audiences during the 2000 Census.
Currently, the advertising campaign is based on the theme “This is your future—don’t leave it ” and stresses how responding to the census questionnaire benefits one’s community. This same general theme was also used for the 1990 Census. However, Bureau officials are hoping that the paid-advertising strategy will allow more effective ad placement as well as more precise targeting of specific messages to particular groups compared to the pro bono promotional services that were used in previous censuses.
The focus group results give an indication of the task Y&R faces in developing an effective campaign. While some participants in the sessions we observed were generally aware of the census and its purpose, others were unfamiliar with the census. Furthermore, while some participants reacted positively to the test advertisements they were shown, others were more cynical and remained unconvinced that they should respond to the census. Cultural and other demographic differences will likely reinforce the difficulties confronting the Bureau. Moreover, as demonstrated by the results of the 1990 Census, simply raising public awareness is insufficient. The public must also be motivated to return their questionnaires in the face of a long-term decline in the mail response rate. The Bureau’s research found that, although about 93 percent of the public was aware of the 1990 Census, the mail response rate was only 65 percent. This was 10 percentage points lower than the mail response rate in the 1980 Census.
section_title: The Bureau’s Ability to Staff the 2000 Census Is Unclear
paragraphs: Recruiting, hiring, training, and retaining a workforce sufficient to accomplish the decennial census is undoubtedly one of the government’s greatest human resource challenges. Although the Bureau has a number of initiatives aimed at making it easier to recruit and retain census workers, a number of uncertainties surround the Bureau’s efforts and their likely success in 2000.
subsections: section_title: The Bureau Has Developed Several Initiatives to Address Potential Staffing Difficulties but Could Still Encounter Problems in 2000
paragraphs: The Bureau estimates that, under its current design, it will need to fill about 295,000 office and field positions to carry out various activities for the 2000 Census. Key positions include census takers (enumerators), crew leaders, field operations supervisors (supervisory enumerators), and clerical office workers. To fill this many positions, the Bureau estimates it will need to recruit some 2,632,600 applicants because, for a variety of reasons, most applicants never make it through the employment process. Some eventually take other jobs, others do not report for testing or training, while still others fail background checks and employment tests. Overall, the Bureau says that for every 10 applicants, only 1 is ultimately hired.
Aside from the sheer number of applicants needed, staffing the census is difficult because most census jobs are temporary and do not offer such benefits as health insurance, life insurance, sick or annual leave, retirement plans, or child care. However, census workers are eligible for unemployment and worker compensation. Also, the majority of the office and field positions must be filled during the short window when nonresponse follow-up operations are conducted. As currently planned, for the 2000 Census, this window will occur during a 6-week period beginning the end of April 2000 and lasting through early June. During that time, the Bureau has set a staffing goal of 121,000 people.
As we reported in our previous work, during the 1990 Census, the Bureau encountered a number of staffing problems that adversely affected both the cost and quality of the census. For example, almost one-half of the Bureau’s district offices were unable to meet the 1989 recruiting goal of four applicants for every field position for precanvassing operations, during which census workers verified the completeness and accuracy of address lists. As a result, 14 percent of the 109 district offices finished the operation at least a week behind schedule. Furthermore, three of the Bureau’s regional offices reported that over 70 percent of their district offices were understaffed as of early May 1990—a peak period for following up on nonresponding households.
subsections: section_title: The Bureau Plans Several Initiatives to Ensure It Will Have an Adequate Workforce, but Staffing the 2000 Census Could Still Be Difficult
paragraphs: To address the potential problem of recruiting, training, and retaining almost 300,000 staff to fill office and field positions, the Bureau plans to use several recruitment and compensation strategies. For example, during the 1990 Census, the Bureau focused its recruitment efforts on unemployed individuals; however, during the 2000 Census, the Bureau plans to target its hiring toward individuals seeking “part-time, part-time” employment (i.e., individuals who are already employed but who are seeking additional work (moonlighters)). Retirees wishing to supplement their income are also a focus of the Bureau’s hiring efforts.
Bureau officials said that moonlighters and retirees are more likely to find the temporary, intermittent weekend and evening work of an enumerator attractive. The officials also said such individuals may be more likely than unemployed individuals to have existing health insurance and other benefits as well as the needed job skills to do the work. Furthermore, they might be in a better position to work evenings and weekends when more people are likely to be home, thus making the Bureau’s nonresponse follow-up efforts more efficient.
To further reduce recruitment and retention problems such as those experienced in the 1990 Census, the Bureau plans to implement a wage-setting program, key features of which include locality-based pay and bonuses for exceeding production. The Bureau’s compensation program is based on recommendations contained in a June 1997 interim report from Westat, which is a private consulting firm. The Bureau asked Westat to analyze factors (pay, working conditions, demographic and economic trends) that appeared to have contributed to the success or failure of hiring that took place in the 1990 Census. The Bureau plans to set the wages it pays census workers at 60 to 80 percent of the prevailing wage rate for the local area as reported by the Bureau of Labor Statistics. According to Bureau projections, the resulting pay rates would range from $8 to $15 per hour.
The Bureau also plans to pay Census employees bonuses for exceeding production and quality standards and for completing assigned work for selected operations. For example, enumerators who obtain completed census questionnaires at the rate of 1.5 cases per hour during nonresponse follow-up operations are to receive a bonus of about $2.00 per case.
To expand the potential applicant pool, the Bureau has taken steps to make it easier to hire federal employees and individuals on public assistance without affecting their pay and benefits. For example, the Bureau has the authority to waive pay/retirement reduction requirements for federal military and civilian retirees who work on the 2000 Census. The Bureau is also seeking assistance from the Office of Personnel Management to remove certain administrative barriers to make it easier to hire current federal employees to work on the census. For individuals on public assistance, for example, the Bureau requested the Department of Housing and Urban Development to exclude the amounts earned by census enumerators when calculating annual income for eligibility in public and assisted housing programs. According to the Bureau, the Department approved the request in May 1996.
Despite these initiatives, the Bureau may find that its efforts to adequately staff the 2000 Census could be problematic. On top of the hiring difficulties that the Bureau has encountered during past censuses, if current national trends continue, the Bureau could be facing an extremely tight labor market. In 1990, the overall unemployment rate was 5.6 percent, compared to 4.9 percent in 1997—the lowest since 1973.
Furthermore, if the Bureau is prevented from using statistical sampling and estimation procedures, recruiting and retaining a sufficient workforce for the 2000 Census could be even more difficult. According to the Bureau, if all nonresponding households are followed up by enumerators, an additional 59,000 enumerators would be needed on top of the estimated 300,000 staff. Additional effort would also be required to verify whether households were vacant. If the Bureau is allowed to sample, the Bureau plans to verify 10-percent of the housing units identified as vacant by the Postal Service. According to the Bureau, verifying a 10-percent sample of vacant housing units would require 5,000 enumerators. However, in the absence of sampling for vacant housing units, the Bureau estimates that 25,000 to 30,000 enumerators would be required for this operation. These enumerators are in addition to those required for nonresponse follow-up.
section_title: The Bureau’s Recruitment Strategy Was Based on Limited Data
paragraphs: We are also concerned that the Bureau’s decision to focus its recruitment efforts on moonlighters and retirees was based on limited data. Bureau officials told us that the rationale for its decision was based, in part, on the Bureau’s experience during the 1995 Census Test, when informal briefings were held with enumerators. The officials noted that the Bureau did not do a formal evaluation of the composition of the workforce for the 1995 Census Test.
The Bureau noted that its recruitment strategy decision was also based on research conducted by Westat. However, Westat’s study acknowledges the limitations of the data, which were partly based on Westat’s own hiring practices and those of similar firms. Westat noted that most of its recruiting, training, and surveying is conducted in the evening and on weekends using employees that have other jobs or family responsibilities during regular business hours.
However, we believe that Westat’s implicit assumptions that its workforce (1) consists of similar kinds of employees who would be interested in census work and (2) is comparable nationwide, in the absence of any other data, may or may not be valid. For example, as noted by a Westat official, unlike census enumerators who are employed in all parts of the country to collect information by going door-to-door, the majority of Westat’s interviewers work in the Washington, D.C., area, and conduct interviews by telephone. Further, although Westat conducted a detailed analysis of the nonresponse follow-up workforce used during the 1990 Census in Pennsylvania, it lacked certain data relevant to its recommended recruitment strategy. For example, the resulting report recommended further analysis to determine such key relationships as how well prior employment and earnings predict recruitment and retention and how productivity varies as a function of employment patterns.
section_title: The Dress Rehearsal Will Not Validate the Bureau’s Staffing Initiatives
paragraphs: According to Bureau officials, staffing the dress rehearsal activities that have occurred to date has gone well. The Bureau was able to hire enough people to adequately staff its operations. Nevertheless, the demand for temporary census workers thus far has been small, and more generally, the Bureau’s experience gained during the dress rehearsal does not and cannot provide an adequate test of the effectiveness of the Bureau’s staffing strategy for the 2000 Census.
According to officials in the Bureau’s Seattle Regional Office, the Bureau has been able to adequately staff its operations for the Sacramento dress rehearsal. They noted that the quality of the people hired, to date, has been good, and has included people with master’s degrees and retired corporate executives. Similarly, officials in the Bureau’s Charlotte Regional Office stated that they have had no difficulty attracting and hiring the necessary staff.
Nevertheless, only a relatively small amount of hiring has taken place. As of January 1998, Bureau workforce data indicate that, for calendar year 1997, a total of only 488 individuals were hired at the three dress rehearsal sites. Of the 488 individuals hired, 352, or about 72 percent, worked at various locations at the Columbia site, primarily conducting address listing operations. Sacramento dress rehearsal operations accounted for 127 of the 488 hires. These individuals were involved with address development operations. Very limited hiring was done in calendar year 1997 to staff dress rehearsal operations at the Menominee County site, where nine individuals were hired to complete address listing activities at this location. Many more individuals will be needed after Dress Rehearsal Census Day for nonresponse follow-up operations and statistical sampling procedures in Sacramento.
However, the dress rehearsal is not a good test of the Bureau’s staffing strategy for the 2000 Census. First, the labor market at the dress rehearsal sites is not necessarily representative of what the Bureau will encounter nationally in the 2000 Census. For example, according to Bureau officials, recruiting at the Sacramento site was made somewhat easier by the closure of a nearby air force base, which added more people to the potential applicant pool.
Moreover, as the demand for census workers increases during the dress rehearsal for such operations as nonresponse follow-up, the Bureau can compensate for any local hiring difficulties by shifting employees to where they are needed most, or even recruiting outside of the dress rehearsal jurisdictions. In the 2000 Census, with each community being called upon to supply the necessary resources to carry out census activities, the Bureau will be less able to use such measures.
section_title: The Bureau’s Plans for Sampling and Statistical Estimation Face a Number of Challenges
paragraphs: Declining response rates have posed problems for the Bureau since it began its mail-out/mail-back procedure in 1970. Hundreds of thousands of additional enumerators must be hired to collect census information from an increasing number of nonresponding households. To reduce its nonresponse follow-up workload, the Bureau plans to sample nonresponding households for the 2000 Census. The Bureau has also designed a procedure called Integrated Coverage Measurement (ICM), which involves taking a separate sample after the nonresponse follow-up is completed to make adjustments to the census counts. However, these activities face several methodological, technological, and quality control challenges. The 1998 dress rehearsal may provide experience useful to addressing these challenges, but others may surface as time goes on.
subsections: section_title: The Bureau Plans to Use Statistical Methods to Reduce Its Nonresponse Follow-Up Workload
paragraphs: In 1990, the Bureau sent Census questionnaires to most households in the United States, asking that household members complete the questionnaire and mail it back to the Census Bureau. The Bureau hired enumerators to visit and collect data from the 35 percent of households that did not return a questionnaire. This follow-up effort proved costly. About $560.3 million of the total $2.6 billion allocated for the 1990 Census (about 50 percent of the data collection budget) was spent on nonresponse follow-up. Furthermore, the effort ran substantially behind schedule because a number of housing units were particularly difficult to contact and required repeated attempts to enumerate.
Because of the difficulty in enumerating these housing units, on occasion, the Bureau had to rely on “last resort” and close-out data collected from neighbors or other knowledgeable sources for those housing units that could not be personally contacted. Such data has been shown in the past to be less accurate than data collected directly from housing unit members because external sources cannot be expected to have complete information about a housing unit or to know if its members were counted elsewhere. For example, in our 1992 report, we found that these procedures may have contributed to the decline in the quality of 1990 Census data because the Bureau accepted less complete responses and information from nonhousehold respondents. Bureau data from the 1990 Census show that last resort and close-out procedures were used to gather information on 6.6 percent of the persons enumerated during nonresponse follow-up. We reported that these procedures were used most heavily in urban areas and among minority populations. Last resort and close-out data accounted for an average of 20 percent of the households in large-city census offices and 4.9 percent of the non-Hispanic Black enumerations (compared to 2.3 percent of the non-Hispanic White enumerations).
For the 2000 Census, the Bureau anticipates that the mail response rate will be 66.9 percent, which is essentially the same as the 65 percent mail response rate achieved in the 1990 Census. Rather than following up with visits to every nonresponding housing unit, the Bureau has proposed a strategy to enumerate remaining housing units on the basis of a sample of nonresponding housing units in each census tract. After the end of the mail response phase, the Bureau will select a sample that would result in a response rate of at least 90 percent of the housing units in each tract. Sample data will then be used to impute the characteristics of the remaining 10 percent of the housing units. To ensure equitable treatment of census tracts with high response rates, all tracts with an initial response rate of at least 85 percent in the mail-back phase will be sampled at a rate of 1 in 3, which will result in more than 90 percent of the housing units in these tracts being enumerated. The Bureau believes its plan will reduce costs and the time necessary to conduct follow-up of nonresponding housing units.
For the dress rehearsal in Sacramento, 2000 Census procedures will be used for nonresponse follow-up. A systematic sample, sorted by address, will be selected in each census tract once the mail-back phase of enumeration is completed. If a census form is mailed back after the nonresponse follow-up sample has been selected, the data from the mailed form will be used for that household, and forms sent in by households will continue to be accepted until the end of nonresponse follow-up data collection.
section_title: The Bureau Plans to Adjust for Enumeration Errors
paragraphs: Since the 1940 Census, the Bureau has conducted a variety of studies to measure inaccuracies in census population counts. Methodologies have varied over the years—Post-Enumeration Surveys (PES); demographic analyses of official records such as birth and death certificates; and matches to the Current Population Survey. Regardless of the method used, a net undercount has been measured for each census since 1940. During the 1990 Census, the Bureau conducted a PES of 167,046 housing units to measure inaccuracies in the population count. However, then-Secretary of Commerce Robert Mosbacher ruled that evidence for adjusting census data, on the basis of the 1990 PES design, was inconclusive and unconvincing. Consequently, the Bureau used PES results only to evaluate the quality of census data, not to revise its final 1990 census numbers. To measure inaccuracies in the population count for the 2000 Census, the Bureau has proposed the ICM, which corrects several problems identified in the 1990 PES and which relies on a much larger sample. The Bureau expects to use the ICM results to produce a “one-number census.”
Based on the PES, the net national undercount (the number of people omitted minus the number overcounted) in 1990 is estimated to have been 4 million people, or 1.6 percent of the population. The net undercount reflects 10 million people who were not counted or who were omitted from the census in their proper place of residence (i.e., omissions) and 6 million people who were either enumerated more than once, were ineligible to be counted, or were counted at a locations other than their correct place of residence (i.e., erroneous enumerations). While the net undercount in 1990 was 4 million people, the differential undercount of minorities and other hard-to-enumerate groups and areas was more problematic. For example, in the 1990 census, the rate of undercount was over four times higher for African Americans than for non-African Americans. Evaluations of the 1990 Census have indicated that attempts to add additional counting activities did not overcome this problem.
The ICM is designed to improve the accuracy of the population counts for a variety of segments of the population and would be the last phase in census data collection, following receipt of mail-out questionnaires and participation in nonresponse follow-up. The Bureau is to draw a sample of 750,000 housing units throughout the United States for the ICM, based on an address list created independently from that prepared for the census enumeration. Enumerators are then to be sent to these housing units to interview residents and obtain the characteristics of individuals living there. In blocks selected for the ICM, sample-based, nonresponse follow-up will not be used. Both the census enumeration and the ICM will attempt to collect information from all housing units in blocks selected for ICM. The Bureau will then compare data collected by the ICM and by the Census enumeration in ICM blocks to determine the degree to which individuals enumerated by one data collection were also detected by the other. Information from this process is to be used to estimate the extent to which people were correctly enumerated, missed, or counted in error for the entire census. The Bureau would estimate the correct population for entire geographic areas, as well as for specific subpopulations as defined by characteristics such as age, sex, tenure, race, and ethnic origin.
section_title: Sampling and Statistical Estimation Designs Face Methodological, Technological, and Quality Control Challenges
paragraphs: In our July 1997 report, we noted that sampling for nonresponse follow-up could reduce costs and save time, while the ICM could improve the accuracy of the population totals. Nevertheless, the Bureau faces a number of operational challenges in implementing its sample-based, data collection activities.
First, as currently designed, the schedules for completing certain phases of the census—including mail response, nonresponse follow-up, and ICM—are compressed when compared to the 1990 Census. In 1990, the Bureau allowed 4 weeks from Census Day for mail response before beginning nonresponse follow-up. In 2000, the Bureau also plans to allow 4 weeks from Census Day for mail response. In 1990, nonresponse follow-up was scheduled to last 6 weeks but in some locations lasted 14 weeks. For the 2000 Census, the Bureau will again allow 6 weeks for completion of nonresponse follow-up. In 1990, PES data collection and reconciliation were not completed until January 4, 1991, while for the 2000 Census, the Bureau plans to perform the same tasks for 5 times the number of households by the end of September 2000. Furthermore, ICM activities from data collection to completion of ICM follow-up interviewing and matching are scheduled to last 5 months. The PES performed after the 1990 census used a much smaller sample of 167,046 housing units, yet it required more than 6 months to complete.
Second, given these time constraints, the Bureau may have to limit repeated call-backs to hard-to-enumerate housing units and rely more heavily on other sources for last-resort or closeout census data to complete the nonresponse follow-up phase on time. In the 1990 Census, the Bureau’s reliance on last-resort data varied by location, with large city census offices enumerating an average of 20 percent of all nonresponse follow-up households by last-resort or closeout procedures. The quality of census enumeration data can suffer when the Bureau uses last-resort or closeout data, because other sources, such as landlords, neighbors, or relatives, may not have complete information on the occupants of a housing unit.
Third, the quality of the final census numbers, as adjusted by the ICM, will be highly dependent on accurate and precise ICM measurement of those persons who were properly enumerated and those who were omitted. The Bureau has set a target of a 98 percent completion rate for ICM interviews—which was the rate it was able to achieve for PES during the 1990 Census—to provide the data quality it expects for adjusting the population count.
As noted earlier, completion schedules for ICM activities are compressed when compared to 1990 PES time frames. A potential consequence of failure to complete ICM on schedule is that the completion rate for ICM data may fall below the expected 98 percent, which could adversely affect the quality of ICM data and therefore the final adjusted census counts.
More specifically, a lower ICM completion rate may result in some segments of the population being underrepresented in ICM data. Comparisons of housing units interviewed during the 1990 PES indicate that it was more difficult to complete interviews in housing units composed of singles, minorities, and renters. Thus, if the ICM completion rate is below the 98 percent target, there is increased risk that these hard-to-count groups will be underrepresented in ICM data, and that ICM-based adjustments may not reflect different segments of the population accurately. Reaching the targeted completion rate for hard-to-enumerate populations may require the Bureau to rely on a larger proportion of last resort and close-out data, which, as noted earlier, are typically of lower quality.
Fourth, difficulties in properly identifying and matching ICM information back to data submitted by ICM households during the enumeration phase of the census may affect the usefulness of ICM data files and estimates. In December 1997, the U.S. Department of Commerce Inspector General reported that software designed to match an ICM sample housing unit to the same census enumeration housing unit will only search within the block to which that housing unit has been assigned. A discrepancy of as little as one digit in the housing unit address between ICM and the census enumeration could result in the housing unit being located in different blocks by ICM and census enumeration, thus precluding a match. To improve the quality of its software development, the Bureau says it is implementing the Inspector General’s recommendations, taking such steps as using peer review and checklists to ensure the proper implementation of sampling and estimation methodology in software.
Housing units selected for ICM whose residents move between Census Day and the beginning of ICM data collection may create additional problems. The failure to locate and interview the same individuals who lived at ICM addresses at the time of the enumeration will lower the ICM completion rate. Tracking individuals who move is important because such individuals may differ in important characteristics, such as family size or marital status, from less mobile individuals. As a fallback, data may be collected from successor households in the housing unit or from other knowledgeable sources outside of the housing unit, but these sources cannot be expected to always have full or accurate information on members of the original household.
Fifth, the Bureau’s implementation of the ICM for the Dress Rehearsal has suffered from a number of quality control problems. During the course of our work, we observed the Bureau’s difficulties in drawing a useable sample of housing units to conduct the ICM for the dress rehearsal. The Bureau initially drew an ICM sample from a list that included commercial addresses that should have been excluded. Inclusion of these commercial addresses rendered the initial sample unusable. The Bureau drew a second ICM sample from a list that excluded these ineligible commercial addresses. However, the Bureau’s specifications for selecting blocks with different racial, ethnic, and renter/owner characteristics for the sample contained a number of inconsistencies. As a result, there were several errors in the number of housing units selected for the different categories of blocks. Because these errors were not detected until relatively late in the sample selection process, we are concerned about the Bureau’s ability to detect and correct problems in a timely manner.
section_title: Conclusions
paragraphs: We believe that, although the Bureau has made progress in addressing some of the problems that occurred during the 1990 Census, key activities continue to face developmental and/or implementation challenges. We recognize that difficulties are to be expected with such a massive and complex undertaking as the census. Nevertheless, with the Decennial Census Day just 2 years away, we find it troubling that the Bureau finds itself facing several ongoing and newly emerging operational challenges. In addition to these operational issues, the continued lack of an agreement between Congress and the administration over the final design of the 2000 Census contributes to the uncertainty surrounding the Bureau’s ability to conduct an accurate and cost-effective census. The risk of a failed census in 2000 will increase as long as the impasse over its final design persists.
The dress rehearsal is the Bureau’s final opportunity for a large-scale, operational test of its overall design. If properly planned, executed, and evaluated, the dress rehearsal could provide the Bureau with valuable information on the feasibility of its current approach and on where corrective action might be needed. With little time remaining until the 2000 Census, a window of opportunity will open only briefly to allow the Bureau to assess the results of the dress rehearsal and take the necessary steps to conduct an accurate, cost-effective census in 2000.
subsections: section_title: Agency Comments and Our Evaluation
paragraphs: The Acting Director of the Bureau of the Census provided written comments on a draft of this report. These comments, along with our response, are summarized below and printed in full in appendix I.
The Bureau agreed that the key census-taking activities we reviewed were facing operational challenges, but expressed confidence that these activities are well under control. The Bureau is concerned that delays and uncertainties caused by the lack of agreement on the Bureau’s planned use of statistical methods have increased the risks to the census. We believe that the successful implementation of key census operations is a prerequisite for a high quality census, regardless of whether or not the Bureau uses statistical methods.
The Bureau also commented on our characterization of specific operational issues. The Bureau generally agreed with the facts as presented with regard to the original design for the address list development methodology and the reengineered design. It also concurred with our assessment that achieving a 99 percent inclusion rate for its address list is one of the greatest challenges for the 2000 Census. However, the Bureau disagreed with our statement that the reengineered design has not been tested. It noted that the methodology is proven and has been used in previous censuses and tests. We agree that the components of the reengineered methodology—canvassing census blocks, working with the Postal Service, and LUCA—have been used and tested, but the Bureau has not used or tested them in concert with each other, nor in the sequence presently designed for use in the 2000 Census. For example, in 1990, block canvassing was done before local government became involved. Further, local governments were not provided address lists, but instead just numbers of addresses in particular census blocks. Moreover, data from the 1990 Census suggests that address list development methodologies may not necessarily be uniform across all jurisdictions or sufficient to meet the Bureau’s goal for the accuracy of its address list.
The Bureau also noted that, as part of its reengineering effort, it has taken steps to improve LUCA, which it agreed did not progress smoothly during the dress rehearsal. These steps include providing additional time for local review, simplifying the process, and suggesting alternative strategies requiring less intensive efforts for jurisdictions having limited resources. However, as noted in the report, we believe that the initiatives will address some, but not all, of the concerns cited by local officials. For example, we remain concerned that the quality of the address lists and maps provided to local governments may limit the effectiveness of local review.
The Bureau agreed with the importance of outreach and promotion activities, but disagreed with our characterization that “results to date of the dress rehearsal  suggest . . . the Bureau may have difficulty achieving its goal of 66.9 percent mail response rate in 2000.” The Bureau stated that the 66.9 percent response rate is derived from extensive testing of its multiple mailing strategy, which includes such innovations as use of first class postage and a reminder card.
Achieving a 66.9 percent response rate is the Bureau’s best-case scenario for the 2000 Census. It depends on many traditional census-taking methods and on new initiatives developed since the 1990 Census, including the multiple mail strategy, an outreach program that includes $100 million in paid advertising, and simplified questionnaire forms. The Bureau’s statement that the 66.9 percent response rate depends primarily on the multiple mailing strategy is inconsistent with its past statements and its approach to building high levels of public cooperation with the census. Thus, we have retained our language that achieving the 66.9 percent mail response goal is dependent upon a successful mailing strategy and an effective promotion and outreach program. All the census-taking methods must work successfully in concert if the Bureau is to achieve its desired response rate.
With regard to the Bureau’s ability to staff the 2000 Census, the Bureau agreed with our assessment concerning the magnitude of the challenge facing the Bureau. However, it disagreed with our characterization that “the Bureau’s recruitment strategy was based on informal and limited data.” Given the nature of today’s society, the Bureau’s decision to focus on evening and weekend hours for conducting nonresponse follow-up interviews appears reasonable. And, as a result of information subsequently provided by the Bureau that shows the additional research on which the Bureau based its recruitment strategy, we agree that the data were not “informal” and have revised the report accordingly. However, our conclusion on the limitations of the data were based on Westat’s own acknowledgment. For example, in one presentation to the Bureau, Westat suggested that “about 847 enumerators will be needed per district office out of a pool of 20,300 workers available in any given month.” Westat stated that this was a “back-of-an-envelope” estimate. In discussing the pool of workers that may be available to be enumerators, Westat stated: “At this time, our best evidence for the view that the success of the  hinges on tapping into the pool of ‘moonlighters’ comes from Westat’s own hiring practices and those of similar firms.”
The Bureau agreed its sampling and statistical estimation plans pose certain challenges and added that a nonsampling census plan would present many of the same challenges as well as many others. The Bureau agreed with our concerns about the tight scheduling of its nonresponse follow-up and ICM operations as well as about quality control issues. The Bureau observed that it will implement a number of initiatives, including its multiple mail strategy, that are designed to improve census data quality. The Bureau also noted that evaluations of the 1990 census, including ours, found that data quality diminishes the farther the collection effort gets from Census Day. Finally, the Bureau said that ICM is the only proven method for reducing both undercounts and overcounts, and the only option for conducting the census within current scheduling constraints.
The Bureau stated that it takes seriously the challenges with regard to quality control, pointing out that it is developing extensive risk mitigation strategies for key areas of the census. It stated that it is improving the quality of software development and using peer review and checklists to ensure the proper implementation of sampling and estimation methodology. The Bureau will be sharing the results of its efforts in the near future. It remains to be seen whether these efforts reduce the risk to the census.
In summary, the Bureau recognized that it faces substantial challenges in conducting an effective census in 2000, but remains confident that it will be successful. We remain concerned, however, that important questions remain unanswered at a disturbingly late stage in the census cycle when the emphasis should be on fine tuning rather than basic revisions to operational plans.