Decennial Census: Preparations for Dress Rehearsal Underscore the Challenges for 2000
Released: 1998-03-26T00:00:00

Letter

Decennial Census: Preparations for Dress Rehearsal Underscore the Challenges for 2000

Mr. Chairman and Members of the Subcommittee: I am pleased to be here today to discuss the U.S. Census Bureau’s preparations and operational plans for its dress rehearsal for the 2000 Census, which is currently under way at three sites: Sacramento, CA; 11 counties in the Columbia, SC, area; and Menominee County in Wisconsin, including the Menominee American Indian Reservation. To the extent that the dress rehearsal mirrors the actual census, the dress rehearsal could foreshadow how well key census-taking activities might work in the decennial, and thus indicate where additional congressional and Bureau attention is needed now to ensure successful results in 2000. My overall point today is that the dress rehearsal, originally contemplated as a concerted demonstration of a well-defined census design for 2000, instead will leave a number of design and operational questions unanswered. These unresolved issues led us in 1997 to raise concerns about the high risk of a failed census in 2000.

At your request, my statement focuses on the progress, if any, that the Bureau has made since July 1997, when we reported that the risk of a failed census in 2000 had increased since we originally designated the 2000 Census as a high-risk area in February 1997. Specifically, we pay special attention to the challenges the Bureau faces in implementing such key census-taking activities as (1) creating a complete and accurate address list, (2) increasing the mail response rate through outreach and promotion, (3) staffing census-taking operations with an adequate workforce, and (4) carrying out its sampling and statistical estimation procedures. These subjects are covered extensively in our report prepared at the request of the Chairman and Ranking Minority Member of the Senate Committee on Governmental Affairs, which we are making available today. I also provide my preliminary observations on the status of the Bureau’s dress rehearsal evaluation program. rehearsal sites, and contacted Menominee officials by telephone; (2) conducted in-person and telephone interviews with local officials on their experiences in reviewing address lists, promoting the census, andrecruiting and hiring census workers; and (3) where applicable, reviewed relevant documents on these activities.

Information on the Bureau’s dress rehearsal evaluation program was obtained by conducting a content analysis of the Bureau’s evaluation proposals and by interviewing cognizant Bureau officials. Since the Bureau has yet to finalize its evaluation plans, our observations should be considered preliminary.

Background

The 1990 Census was the most costly in history, and it produced data that were less accurate than those from the 1980 Census. About 6 million persons were counted twice in the 1990 Census, while 10 million persons were missed—for a total of 16 million gross errors in the count. Of particular concern was the fact that the 1990 Census was more likely to miss minority groups and renters, particularly those living in rural areas.

To address the problems that occurred in 1990, the Bureau redesigned key components of the census, such as procedures for developing a complete and accurate address list, increasing the mail response rate through outreach and promotion, staffing census-taking operations with a capable workforce, and reducing costs and improving accuracy through sampling and statistical estimation. However, Congress has not endorsed the Bureau’s overall design because of its concerns over the validity, legality, and operational feasibility of the Bureau’s statistical sampling and estimation procedures. Because of the significant and long-standing operational and technical challenges that the Bureau faces in taking the census, and the continuing disagreement between Congress and the administration over the use of sampling, in February 1997, we designated the 2000 Decennial Census as being at high risk for wasted expenditures and unsatisfactory results. In July 1997, with still no agreement and uncertainties surrounding the feasibility of some key census operations, we reported that risks of a failed census in 2000 had increased.

Day—April 18, 1998. However, as is the case with the actual census, the Bureau’s dress rehearsal activities span a much wider period of time than this single day. Following the selection of the dress rehearsal sites in July 1996, the Bureau developed preliminary mailing lists and materials for these locations, contacted local governments at the three sites, and conducted staffing activities to hire temporary census employees in those locations. Similarly, after April 18, the Bureau is to develop its census count by conducting the necessary follow-up activities at nonresponding households and completing other fieldwork.

The key to a successful dress rehearsal is making it as much like the decennial census as possible. Thus, according to the Bureau, the dress rehearsal for the 2000 Census should test nearly all of the various operations, procedures, and questions that are planned for the decennial under as census-like conditions as feasible.

However, as an indication of increasing congressional concern over the Bureau’s plans for the 2000 Census, the administration and Congress agreed, as part of a compromise over the Bureau’s fiscal year 1998 appropriation, that the Bureau would use sampling and statistical estimation methods only in the Sacramento site, rather than at all three dress rehearsal sites as it plans to do nationally in 2000. In the Columbia site, the Bureau is to follow up on all nonresponding households just as it did nationwide in the 1990 Census. At the Menominee dress rehearsal site, the Bureau is also to follow-up on all nonresponding households, but is to use sampling and statistical estimation to improve the accuracy of the population count. Although use of the different methods at the dress rehearsal sites invites a comparison of the results, the dress rehearsal is not a test of competing census designs. Geographic, demographic, and possibly other differences among the dress rehearsal locations preclude such a comparison.

Preparations for the Dress Rehearsal Underscore the Challenges to a Successful Census

of a failed census—one on which the nation would have spent billions of dollars and still have demonstrably inaccurate results.

The Accuracy of the Bureau’s Address Lists and Maps Is Uncertain, and Local Reviews May Be Too Sporadic to Greatly Improve Them

Complete and accurate address lists, along with precise maps, are the foundation of a successful census. Accurate addresses are essential for delivering questionnaires, avoiding unnecessary and expensive follow-up efforts at vacant or nonexistent residences, and establishing a universe of households for sampling and statistical estimation. Accurate maps are critical for assigning correct portions of the population to their proper locations—an operation that is the foundation of congressional redistricting.

To build its address list, which is known as the Master Address File (MAF), the Bureau initially planned, in part, to (1) use addresses provided by the Postal Service, (2) merge these addresses with the address file the Bureau created during the 1990 Census, (3) conduct limited checks of the accuracy of selected addresses, and (4) send the addresses to local governments for verification as part of a process called Local Update of Census Addresses (LUCA). However, the Bureau’s analyses of the completeness of the Postal Service’s addresses, when combined with the Bureau’s 1990 addresses for a selected number of locations, have shown that the resulting address list was not always complete. For example, address lists created in 1995 for two test locations did not include from 3.6 to 6.4 percent of the addresses identified through other Bureau operations, such as field verification.

Following these and similar analyses for lists created in 1996, the Bureau concluded in September 1997 that primary reliance on the Postal Service’s and the Bureau’s 1990 address files was not sufficient, and that it needed to redesign its procedures in order to build a MAF for the 2000 Census that, as a whole, is 99 percent complete. Under the new procedures, which are estimated to cost an additional $108.7 million, the Bureau now plans to canvass neighborhoods across the nation to physically verify the completeness and accuracy of the address file for the 2000 Census prior to local address review. While the components of the new approach have been used and tested in prior censuses, the Bureau has not used or tested them either in concert with each other or in the sequence as presently designed for use in the 2000 Census, and does not plan to do so in the dress rehearsal. Consequently, it will not be known until the 2000 Census whether the Bureau’s redesigned procedures will allow it to meet its goal.

Further, the dress rehearsal results to date suggest that LUCA may be too inconsistent and face too many obstacles to systematically verify or increase the accuracy of the MAF. For example, despite the Bureau’s efforts to encourage all local jurisdictions to participate, just 34 of the 60 local jurisdictions involved with the dress rehearsal participated in LUCA. Reasons for the low participation rate included the lack of resources and/or information to review address lists or maps at the local level. Jurisdictions that participated in LUCA said that problems with the level of Bureau assistance, as well as with the accuracy and completeness of the address lists and maps, impeded their review efforts.

Although the Bureau’s reengineered address development procedures call for obtaining earlier assistance from local governments to review addresses and maps, this does not address other problems encountered by local officials in reviewing address lists during the dress rehearsal, such as the unavailability of Bureau assistance and the inconsistent quality of the address list and maps.

The Bureau’s Outreach and Promotion Efforts Face Obstacles That Could Impede Its Ability to Achieve Its Mail Response Rate Objective

To help increase the mail response rate and thus reduce its costly nonresponse follow-up workload, the Bureau plans to partner with local governments and other organizations to raise public awareness of the census. The Bureau expects that its outreach and promotion efforts, combined with other initiatives, such as simplified census questionnaires, should produce a mail response rate of 66.9 percent for the 2000 Census. This is 12 percentage points higher than the 55-percent response rate that the Bureau expects it would achieve without these activities and slightly higher than the 65-percent response rate achieved in the 1990 Census. Nevertheless, the Bureau’s experience thus far during the dress rehearsal suggests that, in 2000, this goal might be difficult to achieve.

According to the Bureau, the success of its outreach and promotion efforts will depend heavily on the effectiveness of the partnerships it hopes to build with state, local, and tribal governments; the private sector; various media; and other organizations. Citing agency policy, the Bureau has said that it is unable to fund local outreach and promotion efforts. It is therefore placing a priority on working with partners because they can help publicize the census, foster participation, and dispel myths, among other activities. community, social service, religious, and other local leaders, CCCs are to help mobilize grassroots promotion efforts. However, not all of the dress rehearsal jurisdictions where the Bureau hoped to establish committees had done so at the time of our review. For example, in South Carolina, of the 11 counties and the City of Columbia participating in the dress rehearsal, just 3 counties and Columbia had active committees at the time of our review. The eight remaining counties either had not started committees or had formed committees that subsequently became inactive.

We found that the operational problems the CCCs were encountering had several sources. Among these were communication difficulties between the CCCs and the Bureau. Four of the six active CCCs we contacted at the three dress rehearsal sites indicated that the Bureau did not set clear expectations for their CCCs, especially when they were first initiated, and/or Bureau guidance and literature had been minimal.

Another element of the Bureau’s outreach and promotion strategy is a paid advertising campaign. In the 1990 Census, the Bureau relied on pro bono public service advertising to get its message across. In October 1997, the Bureau announced that it had awarded its 2000 Census paid-advertising contract to Young & Rubicam, which is a private advertising agency. The Bureau has budgeted about $100 million dollars for this effort, of which about 80 percent has been earmarked for buying advertising in print and broadcast media. Nevertheless, the advertising agency faces not only the familiar task of developing public awareness of the census, but also the greater challenge of motivating people to return their questionnaires in spite of a long-term decline in the mail response rate. While the Bureau found that 93 percent of the public was aware of the census in 1990, the mail response rate was only 65 percent, 10 percentage points lower than it was in 1980.

The Bureau Could Encounter Difficulties Staffing the 2000 Census

For the 2000 Census, the Bureau estimates that it will need to recruit over 2.6 million applicants to fill about 295,000 positions. Aside from the large numbers of people needed, hiring census workers could be difficult because most census jobs are part-time and temporary and do not come with such benefits as health insurance. Consequently, potential applicants may not find census jobs as attractive as alternative work opportunities. pay employees a wage that is based on local rates and to offer productivity incentives. However, if current employment trends continue, the Bureau could find itself recruiting workers in a tighter labor market than prevailed in 1990. Furthermore, the Bureau’s decision to focus its recruitment efforts on moonlighters and retirees is based on informal discussions with census workers during the 1995 Census Test, the hiring practices of private survey research firms, and census workforce studies that may not necessarily be comparable to the 2000 Census situation.

The Bureau’s Sampling and Statistical Estimation Design Faces Several Uncertainties

Declining response rates have posed problems for the Bureau since it began its mail-out/mail-back procedure in 1970. Hundreds of thousands of additional enumerators must be hired to collect census information from an increasing number of nonresponding households. To reduce its nonresponse follow-up workload, the Bureau plans to sample nonresponding households for the 2000 Census. The Bureau has also designed a procedure called Integrated Coverage Measurement (ICM) by which it is to take a separate sample after the nonresponse follow-up is completed to make adjustments to the census counts. However, these activities face several challenges.

For example, it is uncertain whether the Bureau can complete its nonresponse follow-up and ICM operations in the time allotted, considering that in 1990 similar processes took longer even though the amount of work was less. In 1990, the Bureau allowed 4 weeks from Census Day for mail response before beginning nonresponse follow-up. In 2000, the Bureau also plans to allow 4 weeks from Census Day for mail response. In 1990, nonresponse follow-up was scheduled to last 6 weeks, but in some locations lasted 14 weeks. For the 2000 Census, the Bureau will again allow 6 weeks for completion of nonresponse follow-up. In 1990, an operation similar to the ICM was not completed until January 4, 1991, while for the 2000 Census, the Bureau plans to perform the same tasks for five times the number of households by the end of September 2000. other individuals outside of the nonresponding households, a method that in the past has been shown to be less accurate.

A Rigorous Dress Rehearsal Evaluation Program Is Critical to the Ultimate Success of the Census

A properly designed evaluation program that provides information on the cost, performance, required resources, timing of various census operations, and the quality and completeness of census data, is essential for the Bureau to assess the feasibility of its operational plans. We believe that, to be most effective, the evaluation effort needs to begin with a determination of what information will be required to support critical decisions and when that information needs to be available to Bureau and other decisionmakers. However, we are concerned that, with Dress Rehearsal Census Day a little over 3 weeks away, the Bureau’s evaluation program plan is still a work in progress, and that uncertainties surround the Bureau’s approaches and methodologies for carrying out specific studies.

According to the Bureau, its dress rehearsal evaluation program or “report card” is designed to validate plans for the 2000 Census, measure coverage of persons and housing units, and evaluate the completeness and quality of census data. Specifically, it is to consist of a status report to track the performance of key census operations at different points in time during the rehearsal, a quality assurance checkpoint system to monitor key dress rehearsal processes and signal where additional assistance is necessary to ensure operations remain on track, and a series of evaluations to determine how good the census data is in terms of statistical and other quality measures.

The Bureau plans to establish a set of performance standards for measuring success at each site. Such performance standards are to measure, for example, the completeness of the MAF and the effectiveness of the paid advertising campaign. The standards, however, will not be used to measure the operational performance of one site against another. evaluation plans, and the methodologies for some of these evaluations are not sufficiently defined to provide assurances that needed evaluation data will be available on a timely basis. The Bureau continues to complete the methodological details of its evaluations, and plans to issue those details as they are finished.

In summary, although the Bureau has made progress in addressing some of the problems that occurred during the 1990 Census, key activities continue to face operational challenges at a disturbingly late stage in the census cycle when the Bureau should be fine tuning rather than revising its basic operational plans. Moreover, the ongoing lack of an agreement between Congress and the administration over the final design of the 2000 Census has only added to the challenges facing the Bureau. So long as this condition persists, the risk of a failed census in 2000 will continue to increase.

We look forward to supporting congressional oversight of the planning and conduct of the decennial census, and we will continue monitoring the dress rehearsal and the census evaluation program, as well as the Bureau’s preparations for the decennial census.

Mr. Chairman, this concludes my prepared statement. I would be pleased to answer any questions you or other members of the Subcommittee may have.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.