Decennial Census: Preliminary Observations on the Results to Date of the Dress Rehearsal and the Census Bureau's Readiness for 2000
Released: 1998-07-30T00:00:00

Letter

Preliminary Observations on the Results to Date of the Dress Rehearsal and the Census Bureau’s Readiness for 2000

Mr. Chairman and Members of the Committee: I am pleased to be here today to provide an update on the Census Bureau’s dress rehearsal for the 2000 Census and the Bureau’s readiness for carrying out the 2000 Decennial Census. The dress rehearsal, currently under way at three sites—Sacramento, CA; 11 counties in the Columbia, SC area; and Menominee County in Wisconsin, including the Menominee American Indian Reservation—is designed to demonstrate major operations, procedures, and questions that are planned for the decennial census.

At your request, my statement focuses on how key census-taking operations have performed thus far during the dress rehearsal and the implications that may exist for 2000. When we last testified before Congress in March 1998, we noted that, although the Bureau had made progress in addressing some of the problems that occurred during the 1990 Census, key activities faced continuing challenges. The situation today is much the same. On the one hand, certain census activities, such as staffing the dress rehearsal operations, appear to have gone well. On the other hand, measures of other activities, such as the mail response rate, suggest that the Bureau still faces major obstacles to a cost-effective census. Moreover, while the dress rehearsal activities done thus far have demonstrated the Bureau’s general ability to execute the dress rehearsal according to its operational timetable and plan, the important outcome measure—the quality of the data collected—is not yet available. Further, the Bureau’s general ability to conduct the dress rehearsal according to its operational plan, while encouraging, is not necessarily a predictor of success in 2000. Because the dress rehearsal was performed at three sites, the capacity of regional and headquarters offices, as well as a number of essential census-taking operations, could not be fully tested under census-like conditions. is using sampling and statistical estimation methods at the Sacramento site, in accordance with its plans for a sampling census. At the South Carolina site, the Bureau’s procedures are to follow up on all nonresponding households just as it was to do nationwide in the 1990 Census. At the Menominee dress rehearsal site, the Bureau is to follow up on all nonresponding households, but it is also using sampling and statistical estimation to improve the accuracy of the population count.

My comments today are based on our ongoing review of key census-taking operations that could significantly affect the cost and accuracy of the 2000 Census. They include such activities as (1) creating a complete and accurate address list, (2) obtaining a high level of public cooperation through an effective census promotion and outreach effort, (3) staffing census-taking operations with an adequate workforce, (4) processing census data accurately and using technology efficiently and effectively, and (5) carrying out field activities including both nonresponse follow-up and sampling and statistical estimation procedures.

To assess these activities, we (1) made several visits to the dress rehearsal sites and the Bureau’s data capture center in Jeffersonville, IN; (2) observed key census-taking operations; (3) interviewed Bureau headquarters officials, staff from regional and local census offices, and individual enumerators and their supervisors; and (4) reviewed relevant documents and data the Bureau prepared about these operations. To obtain a local perspective on the dress rehearsal, we conducted in-person and telephone interviews with local officials at the three dress rehearsal sites on their experiences in reviewing address lists, promoting the census, and recruiting and hiring census workers.

Because the dress rehearsal is still under way and more comprehensive data on the results of the dress rehearsal are not yet available, our observations today should be considered preliminary and the Bureau’s data are subject to change pending further refinements and analysis.

Dress Rehearsal Experiences Confirm Need for Revised Approach to Developing Address List

One of our long-standing concerns has been the Bureau’s ability to build a complete and accurate address list and develop precise maps. Accurate addresses are critical for delivering questionnaires, avoiding unnecessary and expensive follow-up efforts at vacant or nonexistent residences, and establishing a universe of households for sampling and statistical estimation. Precise maps are essential for counting persons at their proper locations—the cornerstone of congressional reapportionment and redistricting. Bureau maps are also used for certain census-taking operations such as nonresponse follow-up that entails following up on households that fail to mail back a census questionnaire.

To build its address list, which is known as the Master Address File (MAF), the Bureau initially planned, in part, to (1) use addresses provided by the Postal Service, (2) merge these addresses with the address file the Bureau created during the 1990 Census, (3) conduct limited checks of the accuracy of selected addresses, and (4) send the addresses to local governments and Indian tribes for verification as part of a process called Local Update of Census Addresses.

However, as we reported in March 1998, the Bureau concluded in September 1997 that its reliance on postal and 1990 Census addresses to construct its 2000 Census address list would not yield a sufficiently complete and accurate list. The Bureau therefore decided that redesigned procedures were needed in order to generate a MAF for the 2000 Census that, as a whole, was 99 percent complete. Under the revised approach, after local address review, the Bureau plans to verify physically the completeness and accuracy of the address file for the 2000 Census by canvassing neighborhoods across the country. The Bureau expects the new approach will cost an additional $108.7 million. that, based on the 1995 test census results. For the 1995 test census, about 7.7 percent of the census questionnaires were reported to be undeliverable at the Oakland, California test site and 4.5 percent at the Paterson, New Jersey test site.

In addition, the census maps appeared to be of uneven quality and usefulness at the dress rehearsal locations. For example, local census officials in Sacramento and South Carolina said that the census maps were inaccurate and contained a variety of errors, such as streets that were incorrectly placed and named. In both locations, problems with census maps led some enumerators to use commercially available maps rather than those supplied by the Bureau. In Menominee, because of the rural nature of the site, maps were particularly important. Houses generally lacked numbered street addresses, and, as a result, enumerators had to locate them, in part, by using maps. However, Bureau officials told us that while the quality of the Menominee maps is improving over the course of the dress rehearsal, the maps still have problems that make it difficult for enumerators to locate houses.

As I noted, the Bureau recognized that it needed to revise its approach to building the census address list and to improve the quality of its map products. However, the Bureau’s revised approach to developing its address list is not without risk. Although elements of the revised approach have been used and tested in earlier censuses, the Bureau has not used or tested them together, nor in the sequence as presently designed for the 2000 Census. Furthermore, because the Bureau made the decision to change its address list development procedures in September 1997—after major dress rehearsal address list development efforts were already in place—the revised approach was not used during the dress rehearsal. As a result, it will not be known until the 2000 Census whether the Bureau’s redesigned procedures will allow it to meet its goal of a 99 percent complete address list. The Bureau is scheduled to begin its 2000 Census field canvassing address list efforts in August. We will continue to monitor the Bureau’s efforts to build the census address list.

Obtaining Public Cooperation Continues to Be a Challenge for the Census Bureau

55 percent response rate that the Bureau expected it would achieve without these efforts.

The Bureau always finds that mail response rates during census tests, including the dress rehearsal, are lower than those obtained during an actual decennial census, when public awareness of the census is generally much greater. Table 1 shows the anticipated dress rehearsal mail response rates for the three sites and the rates the Bureau actually achieved.

Despite the fact that the Bureau generally met its response rate goals for the dress rehearsal, significant concerns remain about the degree to which the Bureau will be able to meets its mail response goal for 2000. By way of comparison, the 1988 dress rehearsal for the 1990 Census generated mail response rates that ranged from 49 percent to 56 percent for mailout/mailback operations, and 58 percent for update/leave operations.The mail response rate to the 1990 Census was 65 percent—slightly less than the 67 percent response rate that the Bureau hopes for in 2000.

More importantly, the Bureau does not currently plan to use in 2000 a key ingredient of the response rate achieved during the dress rehearsal—a second mailing. According to a Bureau official, concerns about public confusion have contributed to the Bureau’s decision not to use a second questionnaire mailing in 2000. The preliminary results of the dress rehearsal suggest that the Bureau may need to reconsider its decision. At both the South Carolina and Sacramento sites, the Bureau obtained approximately a 7-percentage point “bump” in response rates by sending a second questionnaire to all households located in mailout/mailback areas. According to a senior Bureau offical, this 7 percentage point increase represents real additions to the count and does not include duplicate submissions from households that already had responded.

The Bureau traditionally has found that simply raising awareness of the census is insufficient; through its various outreach and promotion programs, the Bureau must also motivate people to return their questionnaires. The difficulty in doing this was demonstrated during the 1990 Census when the Bureau found that, although about 93 percent of the public was aware of the census, the mail response rate was only 65 percent—10 percentage points lower than the mail response rate to the 1980 Census.

Today, I will highlight two of the more important components of the Bureau’s efforts to build public awareness and cooperation through its outreach and promotion campaign: paid advertising and partnerships and community outreach.

Paid Advertising

With regard to the Bureau’s paid advertising campaign, in October 1997, the Bureau announced it had hired Young & Rubicam, a private advertising agency, to market the census. The advertising campaign is based on the theme “This is your future—don’t leave it ” and stresses how responding to the census questionnaire benefits one’s community. This advertising effort was evident during our visits to the dress rehearsal sites, where we often observed billboards bearing Census 2000 advertising messages, such as “How America Knows What America Needs,” “The Future Takes Just a Few Minutes to Complete,” and “Pave a Road With These Tools.” In convenience stores, we observed signs that told passers-by that “ Gives Life to New Healthcare Centers.” In Sacramento, we observed outdoor advertising in languages appropriate for the neighborhood. The census was also promoted through broadcast and print media, as well as through less traditional methods such as advertisements on shopping bags at a chain of discount stores. $0.35 million for production and media costs for nontraditional advertising $0.23 million for Menominee media costs; and $1.12 million for Sacramento media costs.

Community Outreach

The Bureau’s use of partnership and community outreach activities and, in particular, its use of Complete Count Committees to help promote the census are other key components of the Bureau’s outreach and promotion campaign. According to the Bureau, Complete Count Committees are intended to help the Bureau take the census by, among other activities, planning and implementing a locally-based promotion effort to publicize the importance of the 2000 Census. The committees are to consist of local leaders, such as representatives of government, education, media, community, religious, and businesses organizations. For the dress rehearsal, the Bureau attempted to form committees in Sacramento and Menominee, as well as in the City of Columbia and the 11 surrounding counties participating in the dress rehearsal. The Bureau recommended that the committees could, among other initiatives, form subcommittees to reach specific segments of the population such as senior citizens; sponsor promotional events; obtain commitments from businesses to promote and support the census; provide the Bureau with testing and training space to assist in the employment of enumerators; and work with local media to cover and publicize census activities.

This past spring, the Bureau sent a Complete Count Committee handbook, in which the Bureau described its plan for implementing the Complete Count Committee program for the 2000 Census, to the highest elected officials in about 39,000 local and tribal governments. The handbook suggested a structure for organizing a grassroots outreach campaign and provided an outline and schedule of nearly five-dozen activities that governments could undertake not only to promote the census, but also assist the Bureau in its data collection and enumerator recruiting responsibilities as well. important, what the committees can expect from the Bureau. The Bureau expects that the committees will secure their own funding and will rely on the Bureau for only a very limited amount of direct assistance. For example, at the dress rehearsal site in South Carolina, the Bureau hired two partnership specialists to help mobilize local groups. These specialists had to distribute their time and energy among the City of Columbia and the 11 surrounding counties included in the dress rehearsal—a workload that is consistent with what will be expected in 2000 when the Bureau plans to have 320 partnership specialists in place across the nation.

Our work at the dress rehearsal sites suggests that the effectiveness of the partnership effort was undermined by an apparent mismatch between the Bureau’s expectations of the committees and what the committees could realistically accomplish. In both South Carolina and Menominee, a message we consistently heard from local officials associated with the committees was that they lacked the human and financial resources to promote the census, communication and guidance from the Bureau were insufficient, and Bureau assistance was limited. As a result, Complete Count Committees in some South Carolina counties were never formed, while others became inactive and some local officials expressed confusion and frustration over what was expected.

Local outreach and promotion appeared to go more smoothly in Sacramento. This was likely due in part to the fact that there was only one Bureau partnership specialist in Sacramento assisting only one Complete Count Committee for Sacramento. However, as I have noted, for the 2000 Census, workloads for the Bureau partnership specialists closer to those I have described for South Carolina are more likely be the norm.

Overall, therefore, the dress rehearsal experience suggests that the Bureau needs to ensure that it has realistic expectations about the contributions that Complete Count Committees will be able to make in promoting the census, building the response rate, and assisting the Bureau.

The Dress Rehearsal Has Not Encountered Significant Staffing Problems to Date

as many as 2.6 million applicants, because for a variety of reasons, most applicants never make it through the employment process.

Despite the uncertainties surrounding the Bureau’s ability to staff the 2000 Census, staffing the dress rehearsal appears to have gone better than expected thus far. As shown in table 2, one measure of the success of the Bureau’s staffing efforts, applicants’ acceptance of job offers for nonresponse follow-up (where the demand for employees is greatest), far exceeded the Bureau’s expectations.

Moreover, managers of the local Census Bureau offices at the dress rehearsal sites we spoke to said that the quality of the newly hired employees’ work was typically good. According to Bureau data, at all three dress rehearsal sites, enumerator productivity came very close to the Bureau’s goal of 1.5 nonresponse follow-up cases completed per hour and enumerator turnover appears to have been lower than expected.

The Bureau attributes its apparently successful dress rehearsal staffing efforts to several factors, including a competitive pay plan and aggressive recruitment. Key features of the Bureau’s pay plan include locality-based wages and bonuses for exceeding production targets. In addition, when the Bureau recognized that it was having difficulty recruiting a large enough pool of qualified applicants to fill its needs for nonresponse follow-up and later census operations in South Carolina, the Bureau raised enumerator pay rates from $9.50 per hour to $10.50 per hour effective April 3, 1998. Enumerator pay was $12.50 per hour in Sacramento and $11.25 per hour in Menominee. public libraries. In fact, recruiting literature appeared to be more prevalent than materials that promoted the census itself.

Questions Remain About the Quality of Data Processing and the Use of Technology

Translating data from completed census forms into a useable format represents another challenge for the Bureau. The Bureau plans to have data capture centers process a total of about 1 billion pages of census questionnaires in 99 work days beginning in March 2000. The Bureau plans to take advantage of commercial off-the-shelf hardware and software through its contractor Lockheed Martin, rather than rely on in-house products. During the dress rehearsal, the Bureau is testing the accuracy of the data input by the new scanning equipment and software designed to perform this operation. Bureau officials reported that this operation met all high-priority processing deadlines, despite experiencing system bugs that will need to be addressed before 2000.

The purpose of the dress rehearsal was to test and debug the system in an operational environment in advance of Census 2000. However, additional load testing is still necessary because the system could not be run during the rehearsal at performance levels that will be needed in 2000. During the dress rehearsal, the scanning equipment used to electronically record responses off census forms experienced system crashes due to flaws in the software. To deal with this problem, the Bureau was forced to cut back the number of scanners in operation at any one time. According to Bureau officials, the software subcontractor, is resolving this and other problems through intensive testing, and will have a new version of its software available for further testing in late August.

According to Bureau officials, another problem related to scanning is the frequency at which the scanners needed to be cleaned of accumulated dust. Initially, the Bureau had planned to clean the machines every 2 hours. However, dust accumulated faster than expected, which necessitated a 5-minute cleaning after each 15 minutes of use. Bureau officials said that poor paper quality appears to be one factor that led to the accumulation of dust. The Bureau and the Government Printing Office are studying the problem. computer-generated images—to test the performance of its scanning equipment. Bureau officials believe that sufficient time remains to complete more testing, incorporate lessons learned from the dress rehearsal, and make technology enhancements before Census 2000.

Critical Operations Remain to Be Completed but Field Operations to Date Finished on Schedule

Of the Bureau’s numerous field operations, two of the largest and most logistically challenging under the Bureau’s current design are nonresponse follow-up and a procedure called Integrated Coverage Measurement (ICM), a survey in which residents in a sample of blocks are interviewed. ICM and enumeration data are used in dual system estimation to adjust for coverage errors in the enumeration.

As currently planned, the Bureau is to reduce its nonresponse follow-up workload for the 2000 Census by sampling nonresponding households. By using a sample-based nonresponse follow-up, the Bureau would reduce the time necessary to complete this activity. This in turn would expedite the beginning of ICM data collection, improving the Bureau’s ability to meet the target date for delivery of census data at the end of December. In addition, compressing the nonresponse follow-up data collection period could shorten the average time between census day and visits to households, thereby reducing the likelihood of enumeration errors caused by households that move between census day and nonresponse follow-up.

The Bureau plans to conduct a nationwide ICM. However, as noted earlier, for the dress rehearsal, the Bureau only sampled for nonresponse and is conducting the ICM in Sacramento. In South Carolina, the Bureau procedures are to follow up on all nonresponding households and do a coverage evaluation operation, just as it did nationally in the 1990 Census. At the Menominee site, the Bureau is to follow up on all nonresponding households and additionally, is using the ICM. completed on time at the Sacramento and Menominee sites, and about a week ahead of schedule in South Carolina. We observed that ICM operations began as scheduled in Sacramento. Major ICM field operations are scheduled to last until late August 1998.

The Bureau’s procedures called for it to take additional steps to prevent contamination of the ICM data. According to Bureau officials, these included efforts to separate the management and implementation of the ICM operation from the nonresponse follow-up operation. For example, the ICM operation was administered entirely by the Bureau’s Seattle Regional Office rather than the local census office in Sacramento. Additionally, nonresponse follow-up enumerators were not told which blocks were included in the ICM, and ICM enumerators were told that they could not tell anyone which blocks had been assigned to them.

The quality of the dress rehearsal data, as measured by the extent to which it is complete and accurate, is still to be determined. With the ICM still in progress, the full results of the ICM will not be known for several months. Moreover, a key question for which information is not yet available is the degree to which the Bureau had to rely on proxy responses from neighbors, letter carriers, and others to complete its nonresponse workload in a timely manner. As part of our ongoing work, we will review the quality of the data collected during the ICM and nonresponse follow-up operations, the Bureau’s procedures for maintaining the independence of enumeration and ICM data, and, more generally, the extent to which the Bureau was able to implement its field operations as planned.

In summary, Mr. Chairman, within the constraints and limitations imposed by the dress rehearsal setting, the Bureau to date has shown a general ability to implement the dress rehearsal at the three sites according to its operational timetable and plan. The Bureau has also shown an ability to adapt to changing requirements as demonstrated by such actions as redesigning its address list development procedures to produce a more accurate and complete list and by increasing wage rates in South Carolina to improve recruiting. the population count and the extent to which proxy data are used—are not yet available.

Mr. Chairman, this concludes my prepared statement. I would be pleased to answer any questions you or Members of the Committee may have.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.