United States General Accounting Office

GAO Report to the Committee on

Governmental Affairs,

U.S. Senate

March 1998 2000 CENSUS

Preparations for Dress

Rehearsal Leave Many

Unanswered Questions

GAO/GGD-98-74

GAO United States

General Accounting Office

Washington, D.C. 20548

General Government Division

B-278755 

March 26, 1998

The Honorable Fred Thompson

Chairman

The Honorable John Glenn

Ranking Minority Member

Committee on Governmental Affairs

United States Senate

This report, which was prepared at your request, reviews the status of key activities that the

U.S. Bureau of the Census plans to test as part of its dress rehearsal for the 2000 Census. To the

extent that it mirrors the decennial, the dress rehearsal could foreshadow how well these

activities might work in 2000, and thus indicate where additional congressional and Bureau

attention is needed now to ensure a successful census.

As agreed with the Committee, this report reviews the results to date of the Bureau’s efforts to

create a complete and accurate address list; increase the mail response rate through outreach

and promotion; staff its operations with an adequate workforce; and reduce costs and improve

accuracy through sampling and statistical estimation methods.

We are sending copies of this letter to the Chairman and Ranking Minority Member,

Subcommittee on the Census, House Committee on Government Reform and Oversight; other

appropriate congressional committees; the Secretary of Commerce; the Director, Bureau of the

Census; and other interested parties. Copies will be made available to others on request.

Please call me or J. Christopher Mihm, Associate Director, on (202) 512-8676 if you have any

questions concerning this report. The major contributors to this report are listed in appendix II.

L. Nye Stevens

Director, Federal Management

 and Workforce Issues
Executive Summary

Purpose In February 1997, GAO designated the 2000 Decennial Census as being at

high risk for wasted expenditures and unsatisfactory results.1

 In the

months that followed, the U.S. Bureau of the Census continued with its

preparations for the 2000 Census, presenting its latest plan in a

legislatively mandated report to Congress in July 1997.2

 Separately, GAO in

the same month reported that risks of a failed census in 2000 had

increased.3

Because of their interest in determining what progress, if any, the Bureau

has made in reducing these risks since mid-1997, the Chairman and

Ranking Minority Member, Senate Committee on Governmental Affairs,

asked GAO to review the status of several census-taking activities that are

key to the success of the 2000 Census. The 1998 dress rehearsal is the

Bureau’s last opportunity for an operational test of its overall design of the

2000 Census and to demonstrate to Congress and other stakeholders the

feasibility of its plans. Similar to the 2000 Census, the dress rehearsal

consists of a number of activities spanning a broad period of time.

As agreed with the Committee, the objective of this report is to review the

status of key activities that the Bureau plans to test as part of its dress

rehearsal for the decennial census: address list development, local

outreach and promotion, staffing, and statistical sampling.

Background The decennial census is the nation’s most comprehensive and expensive

statistical data-gathering program. Accurate results are critical because, as

required by the Constitution, decennial census data are used to

reapportion seats in the House of Representatives. Countless decisions

affecting governments, businesses, and private citizens also depend on

census data.

The 1990 Census was the most costly in history, and it produced data that

were less accurate than those from the 1980 Census, leaving millions of

Americans uncounted. GAO, Congress, the Bureau, and others concluded

that the established approach used for taking the census in 1990 had

exhausted its potential for counting the population cost-effectively, and

that fundamental design changes were needed to reduce census costs and

improve the quality of the data collected. In response, the Bureau

1

High-Risk Series (GAO/HR-97-2, Feb. 1997).

2

Census 2000 Operational Plan, Bureau of the Census, July 1997.

3

2000 Census: Progress Made on Design, but Risks Remain (GAO/GGD-97-142, July 14, 1997).

Page 2 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

redesigned key features, such as instituting procedures for developing a

complete and accurate address list, increasing the mail response rate

through outreach and promotion, staffing census-taking operations with a

capable workforce, and reducing costs and improving accuracy through

sampling and statistical estimation. However, Congress has not endorsed

the Bureau’s overall design because of its concerns over the validity and

legality of the Bureau’s statistical sampling and estimation procedures.

In concept, the 1998 dress rehearsal for the 2000 Census should test

almost all of the various operations and procedures planned for the

decennial census under as close to census-like conditions as possible. The

dress rehearsal sites include Sacramento, CA; 11 counties in the Columbia,

SC, area; and Menominee County in Wisconsin, including the Menominee

American Indian Reservation.

Results in Brief The key activities that GAO examined for this report, including address list

development, local outreach and promotion, staffing, and statistical

sampling, are all still facing the developmental and/or implementation

challenges that led GAO in 1997 to raise concerns about the high risk of a

failed census in 2000. For example, the dress rehearsal, which originally

was intended to be a full-scale demonstration of a well-defined census

design for 2000, because of these unresolved issues, will leave a number of

design questions unanswered. This situation has arisen in part because

Congress and the administration have yet to reach agreement on key

aspects of the census design. And, the longer this disagreement continues

and key desion issues remain unresolved, the greater the risk of a failed

2000 Census.

An accurate address list and associated maps are essential for ensuring

that households receive census questionnaires and for establishing the

universe of housing units for sampling and statistical estimation

procedures. Accurate maps are critical for locating households for

congressional reapportionment and redistricting. The Bureau’s initial plan

for creating the address list and maps for the 2000 Census called for

(1) combining addresses provided by the U.S. Postal Service with the

Bureau’s 1990 Census address file and mapping database and

(2) submitting these addresses and maps to local governments for their

review. However, the Bureau has since concluded that the resulting

address file might not meet its goal of being 99 percent complete. Because

of this, in September 1997, the Bureau announced plans to “reengineer” its

process for creating the address file for the 2000 Census, in part by hiring

Page 3 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

temporary staff to physically verify the accuracy of all addresses by

canvassing streets. However, this reengineered approach has not been

tested because of a lack of resources, and will not be tested because the

Bureau does not have sufficient time to do so. Thus, it is not known if the

Bureau’s reengineered procedures will allow it to achieve its goal of

having a 99 percent complete address list for the 2000 Census.

Moreover, local review of address list information did not go smoothly at

the dress rehearsal sites. Many jurisdictions did not participate because

they said they lacked the resources, while participating jurisdictions

encountered problems, such as the level of Bureau assistance, that

impeded their review. As part of its reengineering effort, the Bureau plans

to seek earlier local government review of addresses from the Postal

Service and the Bureau’s 1990 Census address file. However, the Bureau’s

approach still will not address some of the problems encountered by local

officials in their reviews.

To boost the questionnaire mail response rate and thus reduce its costly

nonresponse follow-up workload, the Bureau plans to partner with local

governments, schools, and other community organizations to promote the

census and has initiated a paid-advertising campaign that is being

developed by a private contractor. However, during the dress rehearsal,

inconsistent local participation in grassroots promotion efforts, and

limited local and Bureau resources, raise the risk that the Bureau may not

be able to rely on these initiatives to play a significant role in motivating

public response and achieving the Bureau’s goal of a 66.9 percent mail

response rate in 2000. Furthermore, the paid-advertising campaign faces

the dual challenge of building census awareness and encouraging people

to return their questionnaires in spite of a number of demographic,

cultural, attitudinal, and other barriers.

The Bureau estimates that it will need to recruit over 2.6 million applicants

to fill about 295,000 office and field positions. However, several

uncertainties surround the Bureau’s ability to accomplish this goal. In

addition to the sheer number of people needed, most census jobs are

part-time and temporary, provide few benefits, and thus may not be as

attractive to potential applicants as other types of work. Although the

Bureau has taken steps to address these staffing obstacles, such as basing

pay on local wage rates, if current employment trends continue, the

Bureau could find itself competing for workers in a tight labor market.

Furthermore, the Bureau’s decision to focus its recruitment efforts on

individuals who are currently employed and are seeking additional work

Page 4 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

(“moonlighters”) and retirees is based on limited data that may not be

comparable nationwide. Likewise, the dress rehearsal does not and cannot

provide an adequate test of the effectiveness of the Bureau’s staffing

strategy for the 2000 Census. Although Bureau officials have noted that the

dress rehearsal operations that have occurred to date have been

adequately staffed, employment conditions at the dress rehearsal sites may

not be representative of what the Bureau will experience nationally in

2000.

The Bureau’s plans for the 2000 Census include using sampling and

statistical estimation techniques to reduce the nonresponse follow-up

workload and using the Integrated Coverage Measurement (ICM), which is

a procedure designed to improve the accuracy of the census population

count by reconciling the original census counts with data obtained from an

independent sample of housing units, and using the results to adjust the

census. However, the Bureau’s plans face a variety of operational

challenges. These challenges include methodological and technological

issues such as conducting the census under a more compressed schedule

compared to that of the 1990 Census, obtaining a high completion rate for

the sample-based data collection activities, and overcoming limitations in

the software used to match data during the ICM. Several missteps by the

Bureau in drawing a sample for the ICM have raised GAO’s concerns about

the Bureau’s quality control procedures.

Principal Findings

The Accuracy of the

Bureau’s Address Lists and

Maps Is Uncertain, and

Local Reviews May Be Too

Sporadic to Greatly

Improve Them

To build its address list, which is known as the Master Address File, the

Bureau planned, in part, to (1) use addresses provided by the Postal

Service, (2) merge these addresses with the address file the Bureau

created during the 1990 Census, (3) conduct limited checks of the

accuracy of selected addresses, and (4) send the addresses to local

governments for verification.

However, the Bureau’s analyses of the completeness of the Postal

Service’s addresses, when combined with the Bureau’s 1990 addresses for

a selected number of locations, have shown that the resulting address list

was not always complete. For example, address lists created in 1995 for

two test locations did not include from 3.6 to 6.4 percent of the addresses

identified through other Bureau operations, such as field verification.

Page 5 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

Following these and similar analyses for lists created in 1996, in

September 1997, the Bureau concluded that primary reliance on the Postal

Service’s and the Bureau’s 1990 address files was not sufficient, and that it

needed to reengineer its process to achieve an address file for the 2000

Census that, as a whole, is 99 percent complete. Under the new

procedures, the Bureau now plans to canvass neighborhoods across the

nation to physically verify the completeness and accuracy of the file for

the 2000 Census. The reengineered approach is estimated to cost an

additional $108.7 million. While the components of the new approach have

been used and tested in the past, the Bureau has not used or tested them in

concert with each other, nor in the sequence as presently designed for use

in the 2000 Census, and does not plan to do so in the dress rehearsal.

Consequently, it will not be known until the 2000 Census whether the

Bureau’s redesigned procedures will allow it to meet its goal.

The dress rehearsal results to date suggest that local participation may be

too inconsistent and face too many obstacles to consistently verify or

increase the accuracy of the address file and maps. For example, despite

the Bureau’s efforts to encourage all local jurisdictions to participate, just

34 of the 60 local jurisdictions involved with the dress rehearsal

participated in the verification effort. Reasons for the low participation

rate included the lack of resources and/or information to review address

lists or maps at the local level. Jurisdictions that did participate in this

effort identified problems with the level of Bureau assistance as well as

with the accuracy and completeness of the address lists and maps, which

impeded their review efforts. The Bureau’s reengineered address

development procedures call for obtaining earlier assistance from local

governments to review addresses and maps. However, this does not

address all of the problems encountered by local officials in reviewing

address lists during the dress rehearsal, such as the availability of

assistance and the quality of the address list and maps. (See ch. 2.)

The Bureau’s Outreach and

Promotion Efforts Face

Obstacles That Could

Impede Its Ability to

Achieve Its Mail Response

Rate Objective

To help increase the mail response rate and thus reduce its costly

nonresponse follow-up workload, the Bureau plans to partner with local

governments and other organizations to raise public awareness of the

census. However, implementation difficulties could adversely affect the

Bureau’s ability to achieve its goal of a 66.9 percent mail response rate in

2000.

Complete Count Committees, which are critical components of the

Bureau’s local outreach and promotion efforts, are to consist of elected,

Page 6 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

business, community, social service, religious, and other local leaders who

are to help mobilize grassroots promotion efforts. However, GAO found

that not all of the dress rehearsal jurisdictions that the Bureau hoped

would establish committees had done so. For example, in South Carolina,

of the 11 counties and the City of Columbia participating in the dress

rehearsal, just 3 counties and Columbia had active committees at the time

of GAO’s review. The eight remaining counties either had not started

committees or had formed committees that subsequently became inactive.

According to local officials, vague guidance and expectations from the

Bureau played a role in the effort’s mixed results. GAO also found that local

and Bureau resources may be too limited to organize an effective local

outreach campaign through the Complete Count Committees. Officials

from five South Carolina counties and the City of Columbia indicated that

resources to carry out outreach and promotion activities were limited,

while Bureau personnel might be stretched too thinly. Bureau officials

noted that the Bureau made a conscious policy decision not to fund local

governments’ outreach and promotion activities because the Bureau did

not see itself as a funding organization. Instead, the Bureau hopes to assist

local governments through its partnership specialists.

Moreover, although the Bureau has contracted with a private firm to

conduct a multimedia, national and locally targeted advertising campaign,

the firm faces not only the familiar task of developing public awareness of

the census, but also the greater challenge of motivating people to return

their questionnaires in spite of a long-term decline in the mail response

rate. While the Bureau found that 93 percent of the public was aware of

the census in 1990, the mail response rate was only 65 percent,

10 percentage points lower than it was in 1980. (See ch. 3).

The Bureau Could

Encounter Difficulties

Staffing the 2000 Census

For the 2000 Census, the Bureau estimates that it will need to recruit over

2.6 million applicants to fill about 295,000 positions. Aside from the large

numbers of people needed, hiring census workers could be difficult

because most census jobs are part-time and temporary and do not come

with such benefits as health insurance. Consequently, potential applicants

may not find census jobs as attractive as alternative work opportunities.

To address this staffing challenge, the Bureau plans to focus its

recruitment efforts on moonlighters and retirees. The Bureau also plans to

pay employees a wage that is based on local rates and offer productivity

incentives. However, if current employment trends continue, the Bureau

Page 7 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

could find itself recruiting workers in a tighter labor market than prevailed

in 1990. Furthermore, the Bureau’s decision to focus its recruitment efforts

on moonlighters and retirees is based on informal discussions with census

workers during the 1995 Census Test, the hiring practices of private survey

research firms, and census workforce studies that may not necessarily be

comparable to the 2000 Census.

According to Bureau officials, the Bureau has been able to adequately staff

the dress rehearsal operations that have occurred to date. However, the

dress rehearsal is not a good test of the Bureau’s staffing strategy for 2000.

The labor market at the dress rehearsal sites is not necessarily

representative of the one the Bureau will encounter nationwide in 2000.

Also, while the Bureau can compensate for any staffing shortages at the

dress rehearsal sites by shifting employees or recruiting outside of the

dress rehearsal jurisdictions, the Bureau’s ability to take such measures in

2000 will be more limited because the demands of taking the census

nationally will reduce the availability of such individuals. Finally, if the

current debate over the use of sampling is resolved by deciding to follow

up on all nonresponding households, the Bureau estimates that an

additional 59,000 enumerators would be needed to do this follow up, and

an additional 25,000 to 30,000 enumerators would be needed to verify

vacant housing units—which would add to potential recruiting difficulties.

(See ch. 4.)

The Bureau’s Sampling and

Statistical Estimation

Design Faces

Methodological,

Technological, and Quality

Control Challenges

For the 2000 Census, the Bureau anticipates that the proportion of

households returning census questionnaires will be about

66.9 percent—about the same as in the 1990 Census. Rather than following

up on all remaining households as it did in 1990, the Bureau plans to

conduct a sampling of nonresponding households. In addition, the

Bureau’s ICM procedures are designed to measure and adjust for any

inaccuracies in the population count.

GAO’s July 1997 report noted that sampling for nonresponse follow-up

could reduce costs and save time, while the ICM could improve the

accuracy of the population totals.4

 However, GAO also noted that to achieve

these objectives, sampling and the ICM would need to be effectively

implemented. Unfortunately, the Bureau’s sampling and statistical

estimation plans face a number of unresolved operational challenges that

could jeopardize their success in 2000. It is uncertain whether the Bureau

will complete its nonresponse follow-up and ICM operations in the time

4

GAO/GGD-97-142.

Page 8 GAO/GGD-98-74 Preparations for Dress Rehearsal
Executive Summary

allotted, considering that in 1990 similar processes took longer even

though the amount of work was less. Uncertainties also surround the

Bureau’s ability to achieve its target of a 98 percent completion rate for ICM

operations. If the ICM completion rate falls below 98 percent, there is

increased risk that ICM-based adjustments to census data may not

accurately reflect different segments of the population. The need to obtain

a high response rate in a short time could require the Bureau to collect

data from neighbors and other individuals outside of the nonresponding

households, a method that has been shown in the past to be less accurate.

Technologically, the software the Bureau plans to use in properly

identifying and matching ICM data to those data collected from the same

household during the census enumeration phase has limitations and could

preclude a match between individuals counted in the census enumeration

and those counted in the ICM. These limitations could affect the precision

of the adjusted population counts.

The Bureau has made several missteps in drawing an ICM sample for the

dress rehearsal, such as inadvertently including commercial addresses.

Because these errors went undetected until relatively late in the sample

selection process, GAO is concerned about the Bureau’s ability to catch and

correct problems during this critical phase of the 2000 Census in a timely

manner. (See ch. 5.)

Recommendations Since plans for the dress rehearsal operations are generally complete, GAO

is not making any recommendations in this report.

Agency Comments

and GAO’S Evaluation

GAO received written comments on a draft of this report from the Acting

Director of the Bureau of the Census. While agreeing that the 2000 Census

still faces operational challenges—building an accurate address list,

motivating people to participate, staffing, and refining its sampling plans

under tight schedule constraints—the Acting Bureau Director expressed

confidence that these activities are well under control and that, through

the adjustments under way, the challenges will be met. GAO remains

concerned, however, that important questions remain unanswered at a

disturbingly late stage in the census cycle when the emphasis should be on

fine tuning rather than basic revisions to operational plans.

The Bureau’s comments are discussed further at the end of chapter 6 and

reprinted in full in appendix I.

Page 9 GAO/GGD-98-74 Preparations for Dress Rehearsal
Contents

Executive Summary 2

Chapter 1 

Introduction

12

The Census Dress Rehearsal Is to Provide a Full-Scale

Demonstration of Most Procedures Planned for 2000

12

A Demonstration of the Bureau’s Plans Is Critical Given Past

Problems

13

Increasing Congressional Concern Has Led to Changes to the

Dress Rehearsal

14

Objectives, Scope, and Methodology 15

Chapter 2 

The Quality of the

Address List Has

Significant

Implications for the

2000 Census

17

The Postal Service’s Address File May Not Be Sufficiently

Accurate or Complete to Meet the Bureau’s Needs

17

Local Reviews May Be Too Sporadic and Contain Too Many

Obstacles to Consistently Confirm or Improve the MAF’s

Accuracy

20

The Bureau’s Reengineering Plans Address Some, but Not All,

Concerns Cited by Local Officials About LUCA

24

Chapter 3 

The Bureau’s Mail

Response Rate

Objective Could Be

Difficult to Achieve

Because of Obstacles

to Implementing Its

Outreach and

Promotion Strategy

26

The Bureau Has Initiated Several Outreach and Promotion

Activities

26

Effectively Implementing the Bureau’s Outreach and Promotion

Activities Could Be Problematic

28

Chapter 4 

The Bureau’s Ability

to Staff the 2000

Census Is Unclear

33

The Bureau Has Developed Several Initiatives to Address

Potential Staffing Difficulties but Could Still Encounter Problems

in 2000

33

Page 10 GAO/GGD-98-74 Preparations for Dress Rehearsal
Contents

Chapter 5 

The Bureau’s Plans for

Sampling and

Statistical Estimation

Face a Number of

Challenges

38

The Bureau Plans to Use Statistical Methods to Reduce Its

Nonresponse Follow-Up Workload

38

The Bureau Plans to Adjust for Enumeration Errors 40

Sampling and Statistical Estimation Designs Face

Methodological, Technological, and Quality Control Challenges

41

Chapter 6 

Conclusions

44

Agency Comments and Our Evaluation 44

Appendixes Appendix I: Comments From the Bureau of the Census 48

Appendix II: Major Contributors to This Report 51

Abbreviations

ACF Address Control File

CCC Complete Count Committee

DSF Delivery Sequence File

FTE full-time equivalent

ICM Integrated Coverage Measurement

LUCA Local Update of Census Addresses

MAF Master Address File

PES Post-Enumeration Survey

Y&R Young & Rubicam

Page 11 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 1 

Introduction

The decennial census is the nation’s most comprehensive and expensive

statistical data-gathering program. An accurate population count is critical

because, as required by the Constitution, census results are used for

allocating seats in the House of Representatives. Census data are also used

for redistricting within states, distributing billions of dollars in federal

funding, guiding investment decisions of the public and private sectors for

the ensuing decade, and providing a baseline for countless other statistical

measurement programs.

Unfortunately, as we noted in a previous report, the nation runs the risk of

a very expensive and seriously flawed census in 2000.5

 To assess the

Bureau of the Census’ progress in minimizing risks, we reviewed the status

of key activities that the Bureau plans to test as part of its dress rehearsal

for the next decennial census: address list development, local outreach

and promotion, staffing, and statistical sampling. To the extent that the

dress rehearsal accurately mirrors the actual census, the results of the

dress rehearsal could indicate what might occur in 2000.

The Census Dress

Rehearsal Is to

Provide a Full-Scale

Demonstration of

Most Procedures

Planned for 2000

According to the Census Bureau, the dress rehearsal for the 2000 Census is

designed to test the various operations, procedures, and questions that are

planned for the decennial under as near census-like conditions as possible.

In July 1996, the Bureau announced that Sacramento, CA; 11 counties in

the Columbia, SC, area; and Menominee County, WI, including the

Menominee American Indian Reservation, had been selected as sites for

the 2000 Census dress rehearsal.

According to the Bureau, Sacramento was selected because its diverse

population will allow the Bureau to test estimation methods designed to

reduce differential undercounts. Sacramento is also a primary media

market, thus enabling the Bureau to test its paid-advertising campaign. The

Columbia site was selected, among other reasons, for its mixed rural and

urban features and because its relatively high proportion of

African-Americans will give the Bureau an opportunity to test methods of

reducing the differential undercount for this population group. The

Menominee site was included to allow the Bureau to test measures aimed

at reducing the undercount of American Indians.

Administratively, the Bureau’s Seattle Regional Office is responsible for

conducting the Sacramento dress rehearsal, while its Charlotte, NC, and

5

GAO/HR-97-2.

Page 12 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 1 

Introduction

Chicago, IL, Regional Offices, respectively, are responsible for the South

Carolina and Menominee dress rehearsals.

An important aspect of the dress rehearsal activities is Dress Rehearsal

Census Day, which is scheduled for April 18, 1998. However, as is the case

with the actual census, the Bureau’s dress rehearsal activities span a much

wider period of time than 1 day. Following the selection of the dress

rehearsal sites in July 1996, the Bureau developed preliminary mailing lists

and materials for these locations, contacted local governments at the three

sites, and conducted staffing activities to hire temporary census

employees in those locations. Similarly, after April 18, the Bureau is to

develop its census count by conducting the necessary follow-up activities

at nonresponding households and completing other fieldwork.

A Demonstration of

the Bureau’s Plans Is

Critical Given Past

Problems

A well-planned and -executed dress rehearsal should provide the Bureau

with an opportunity to demonstrate the feasibility of its design for the 2000

Census. This demonstration is particularly important considering the

problems the Bureau has experienced to date and the controversy that

surrounds its design.

Since 1970, when measured in constant dollars, census costs have

increased faster than inflation, even after allowing for population growth.

In 1990 constant dollars, total census cycle costs were $0.7 billion in 1970,

$1.8 billion in 1980, and $2.6 billion in 1990.6

 The Bureau expects that the

escalating cost of conducting the census will continue; in March 1997, the

Bureau estimated that the 2000 Census will cost at least $4.0 billion in 2000

dollars.

Unfortunately, the nation’s growing investment in the census has not

generated more accurate results. Although the 1990 Census was the most

expensive in history, the results were less accurate than those of the 1980

Census. The Bureau estimated that about 6 million persons were counted

twice in the 1990 Census, while 10 million persons were missed. These

errors were particularly worrisome because the Bureau’s evaluation

showed a persistent differential undercount of minority groups and

renters, particularly those living in rural areas.

The cost of a census is largely influenced by the relative need for the

Bureau to follow up on nonresponding housing units. Since the 1970

6

Constant-dollar value is measured in terms of prices for a base period to remove the influence of

inflation. The resulting constant-dollar value is the value that would exist if prices had remained the

same as in the base period.

Page 13 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 1 

Introduction

Census, the Bureau has used a “mail-out/mail-back” approach for counting

the vast majority of the population whereby the Bureau develops an

address list of the nation’s housing units and mails a questionnaire to each

one. Although households are to mail back completed forms, many

millions do not. This nonresponse requires the Bureau to hire hundreds of

thousands of temporary employees, called enumerators, to follow up on

each nonresponding housing unit by visiting each unit and gathering

census information. The mail response rate to the census questionnaire

has dropped from 78 percent in 1970 to 65 percent in 1990, thus increasing

the Bureau’s costly nonresponse follow-up workload.

Increasing

Congressional

Concern Has Led to

Changes to the Dress

Rehearsal

The Bureau’s plans for the 2000 Census include the use of sampling and

statistical estimation. Initially, the Bureau planned to use these methods at

the three dress rehearsal sites just as it plans to do nationwide in 2000.

Congressional concern over the validity and legality of the Bureau’s

sampling and statistical estimation methods have led to recent changes to

the dress rehearsal. According to Department of Commerce officials, as

part of a compromise between the administration and Congress over the

Bureau’s fiscal year 1998 appropriation,7

 it was agreed that the Bureau

would use sampling and statistical estimation methods only in the

Sacramento site for nonresponse follow-up and to improve the accuracy of

the population count. In the Columbia site, the Bureau is to hire

enumerators to follow up on all nonresponding households, just as it did

nationally in the 1990 Census. At the Menominee dress rehearsal site, the

Bureau is to do a 100 percent nonresponse follow-up but is to use

sampling to improve the accuracy of the population count. Although use of

the different methods at the dress rehearsal sites invites a comparison of

the results, geographic, demographic, and possibly other differences

between the dress rehearsal locations preclude such a comparison.

The Bureau’s fiscal year 1998 appropriation act also allows for expedited

judicial review of sampling and created an eight-member Census

Monitoring Board that is to oversee all aspects of the preparation and

implementation of the 2000 Census and report its findings to Congress.8

7

Departments of Commerce, Justice, and State, the Judiciary, and Related Agencies Appropriations

Act, 1998, P.L. 105-119, Nov. 26, 1997.

8

In February 1998, House Republicans and a public interest law firm each filed a lawsuit challenging

the Bureau’s planned use of sampling.

Page 14 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 1 

Introduction

Objectives, Scope,

and Methodology

Concerned about the risk of a failed census, the Chairman and Ranking

Minority Member, Senate Committee on Governmental Affairs, asked us to

review the status of key census-taking activities that the Bureau is to test

during the 1998 dress rehearsal. In response, we examined the Bureau’s

efforts to (1) create a complete and accurate address list, (2) increase the

mail response rate through local outreach and promotion, (3) staff

census-taking operations with an adequate workforce, and (4) implement

its sampling and statistical estimation methodology. If designed and

implemented as much like the actual census as possible, the dress

rehearsal could yield results indicative of what might occur in 2000.

To examine the status of address list development, local outreach and

promotion, and hiring, we obtained documents that describe the Bureau’s

plans, procedures, and progress for each activity and interviewed Bureau

headquarters officials and, where applicable, regional office officials. We

also reviewed the Bureau’s analysis of missed or duplicated housing units

in the 1990 census; the Bureau’s analyses of missed or duplicate housing

units in its 1995 Census Test conducted in Paterson, NJ, and Oakland, CA;

and its analysis of the completeness of address lists for seven locations

around the nation. This was done to review the accuracy and

completeness of the Bureau’s address list.

To help verify the Bureau’s information and also obtain a local perspective

on the dress rehearsal activities, we (1) made site visits to the Sacramento

and South Carolina dress rehearsal jurisdictions; (2) conducted in-person

and telephone interviews with local officials on their experiences in

reviewing address lists, promoting the census, and recruiting and hiring

census workers; and (3) where applicable, obtained relevant documents

on these activities. Menominee officials were contacted by telephone. To

obtain an indication of the task the advertising contractor faces in

developing an effective promotional campaign, we observed focus group

sessions the contractor scheduled with different population groups as part

of its market research at the dress rehearsal sites.

To examine the Bureau’s efforts to implement its sampling and statistical

estimation methodology, we reviewed Bureau documents related to

sampling and estimation, interviewed Bureau officials who are responsible

for drawing the sample for the dress rehearsal, and reviewed the Bureau’s

methodology for drawing the sample.

We conducted our audit work at the South Carolina and Sacramento dress

rehearsal sites; the Bureau’s Charlotte and Seattle regional offices; as well

Page 15 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 1 

Introduction

as in Washington, D.C., and Dallas, TX, between September 1997 and

February 1998 in accordance with generally accepted government auditing

standards.

We requested comments on a draft of this report from the Secretary of

Commerce or his designee. On March 18, 1998, the Acting Director, Bureau

of the Census, provided us with comments on the draft report. These

comments are presented and evaluated in chapter 6 and reprinted in

appendix I.

Page 16 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

A quality address list—one that is complete and accurate—and precise

maps are the fundamental building blocks of a successful census. Accurate

addresses are essential for delivering questionnaires, avoiding unnecessary

and expensive follow-up efforts at vacant or nonexistent residences, and

establishing a universe of households for sampling and statistical

estimation. Accurate maps are critical for counting the correct portions of

the population in their correct locations—the cornerstone of

congressional reapportionment and redistricting.

For the 2000 Census, the Bureau estimates that it needs to identify and

locate 118 million housing units and to create a database of addresses,

known as the Master Address File (MAF). To build the MAF, the Bureau

originally planned, in part, to (1) use addresses provided by the Postal

Service, (2) merge these addresses with the address file the Bureau

created during the 1990 Census, (3) conduct limited checks of the

accuracy of selected addresses, and (4) send the addresses to local

governments for verification. However, the Bureau’s experience during the

dress rehearsal shows that its procedures for developing an accurate MAF

and maps have weaknesses that, despite the Bureau’s recent improvement

efforts, may not necessarily result in a MAF or maps of sufficient accuracy

to ensure all households receive a census questionnaire. Because of these

procedural weaknesses, the Bureau is “reengineering” how it builds the

MAF by physically verifying all addresses and by encouraging earlier local

government reviews of address lists and maps created from the Postal

Service and the Bureau’s 1990 address file. Although this reengineered

approach may address several problems, the Bureau will not be testing

this new approach before the 2000 Census.

The Postal Service’s

Address File May Not

Be Sufficiently

Accurate or Complete

to Meet the Bureau’s

Needs

To reduce cost and improve the accuracy of the 2000 Census, the Bureau,

in developing the MAF for city-style addresses for the census,9

 originally

planned to combine the Postal Service’s Delivery Sequence File (DSF),

which includes all mail delivery points recognized by the Postal Service,

with the address file created during the 1990 Census. The Bureau also

planned to conduct a limited amount of canvassing in each city to ensure

that certain areas, such as those that have experienced growth, had

accurate and complete addresses or those that have multiunit (e.g.,

apartment) buildings had consistent counts of apartment units. In areas

with mostly non-city-style mailing addresses, such as rural areas that have

9

A city-style address includes a street name and house number (e.g., 22 Oak Street). A significant

characteristic of this type of address is that it is part of an address system, which means that it usually

appears in numeric sequence along a street and reflects parity conventions (i.e., odd numbers appear

on one side of the street and even numbers appear on the other side).

Page 17 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

rural delivery or post office mailing addresses, the Bureau plans to create

a MAF as it did for the 1990 Census by having census enumerators visit all

housing units in the designated areas to identify their physical locations,

and to obtain other data such as each occupant’s name, mailing address,

and telephone number.

On the basis of its analyses of the DSF, the Bureau has concluded that the

DSF, combined with targeted canvassing in city-style address areas, cannot

be relied upon to achieve its goal of having a MAF for the 2000 Census that

has no more than 1 percent net housing unit undercoverage and no more

than 1 percent duplicate housing coverage. One of the major problems

associated with having an inaccurate address list is the cost incurred in

following up on questionnaires that are not mailed back. In the 1990

Census, we reported that the cost to follow up on questionnaires

accounted for 57 percent, or $730 million, of the $1.3 billion spent on data

collection.10 A portion of that amount was caused by inaccuracies in the

address list. For example, according to the Bureau, the cost of

unnecessary follow-up work at housing units that were either vacant or

nonexistent was about $317 million in 1990, a total that included the direct

cost of hiring enumerators and providing training and travel expenses, as

well as the indirect cost of office space and equipment, recruiting, and

administration. To estimate the potential quality of the MAF for the 2000

Census, the Bureau has analyzed address lists created for a few selected

locations around the nation. Following the 1995 Census Test in Paterson,

NJ, and Oakland, CA, the Bureau found that field-verifying addresses (e.g.,

precanvassing), local government review of the MAF, and a final Postal

Service check added 6.4 percent and 3.6 percent of the total addresses on

the MAF for those locations, respectively. Precanvassing and local

government review of the MAF also deleted 9 percent and 6 percent of the

addresses on the MAF for Paterson and Oakland, respectively. The Bureau

also examined the quality of address files created for the American

Community Survey test sites in seven locations.11 The Bureau’s estimates

of missed housing units ranged from a low of 0.0 percent in Rockland

County, NY, to a high of 2.4 percent in Harris County, TX. The Bureau’s

estimates of duplicate or nonexistent housing units ranged from a low of

0.0 percent in Multnomah County, OR, to a high of 11.7 percent in

Rockland County.

10Decennial Census: 1990 Results Show Need for Fundamental Reform (GAO/GGD-92-94, June 9, 1992).

11The American Community Survey is a large monthly household survey that uses mailed

questionnaires, telephone interviews, and visits from Bureau field representatives. It is to be

conducted at additional sites in future years to supplement information obtained in the 2000 Census.

Page 18 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

According to the Bureau, incidents of missed housing units occur because

the DSF is not (1) completely up-to-date in terms of new housing

construction; and (2) uniformly current because the DSF reflects address

changes in certain areas faster than in other areas. Missed housing units

can also occur because the Postal Service may define the delivery points

for mail differently than the Bureau defines separate housing units. This

could occur, for example, when the Postal Service uses one delivery point

for all mail delivered to a multiunit apartment building. Further, erroneous

addresses can be introduced into the MAF when local governments provide

the Bureau with addresses generated from locally developed addressing

systems (e.g., emergency services) that may differ from the addresses

contained in the DSF and used for mail delivery.

According to Bureau officials, in June 1997, the Bureau initiated an effort

to determine whether it could improve the process for building the MAF for

the 2000 Census, and, if so, how. As part of that effort, the Bureau

assembled a team that included tribal, state, and local government

representatives to obtain outside reaction to the Bureau’s original plan to

develop its address list by relying primarily on the DSF and targeted

canvassing, and to discuss new approaches to developing the address list.

The team developed a modified, “reengineered” plan for address list

development that includes, for areas that contain predominantly city-style

addresses, verifying all addresses (100 percent block canvassing) through

field visits, as was done during the 1990 Census. The Bureau estimates that

the reengineered plan will add $108.7 million to its address list

development procedures, which are already budgeted at $357.2 million.

According to the Bureau, the reengineered plan will allow it to achieve its

goal of a MAF that has no more than 1 percent net housing unit

undercoverage and no more than 1 percent duplicate housing coverage.

The Bureau’s initial plan for developing the MAF for city-style addresses

included combining the DSF and the 1990 Census Address Control File

(ACF); conducting limited, targeted canvassing; and local government

review. Its decision to reengineer the MAF building process was based on

Bureau officials’ judgment, following their review of existing data, that the

initial plan would not result in the high coverage rate desired. It is not

known whether the reengineered plan will result in the high accuracy rate

the Bureau wants to attain because the Bureau has not tested this plan to

determine the number and nature of missed or erroneously included

housing units that would occur by using this process. For city-style

addresses in the Bureau’s 1995 Census Test, the Bureau combined the DSF,

the ACF, 100-percent canvassing, local address review, and a Postal Service

Page 19 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

address accuracy check, to compile the MAF. However, Bureau officials

stated that they lacked the resources to determine the final housing unit

undercount and the duplicate housing unit rates in the 1995 Census Test

MAF as was done following the 1990 Census.

Local Reviews May Be

Too Sporadic and

Contain Too Many

Obstacles to

Consistently Confirm

or Improve the MAF’s

Accuracy

Local address review has not progressed smoothly at the dress rehearsal

sites. Many governments did not participate in local review, while some

governments that did participate cited time and resource constraints and

limited assistance from the Bureau as impediments to their reviews.

Under the Bureau’s Local Update of Census Addresses (LUCA) program,

local governments and Indian tribes owning lands are to be given an

opportunity to verify address list information. In 1990, local governments

were given the opportunity to review Bureau counts of housing units, as

well as boundary maps to identify any major discrepancies. The 1994

Census Address List Improvement Act extended the Bureau’s

confidentiality provisions to local governments, and beginning with the

2000 Census, the Bureau plans to provide local government officials with

detailed address information.12 The importance of LUCA in improving the

quality of the Bureau’s MAF was demonstrated during the 1995 Census Test.

During that test, the Bureau found that LUCA alone added, deleted,

corrected, or moved 7.1 percent of the addresses in Paterson and

1.6 percent of the addresses in Oakland.

Time and Resource

Constraints Limited Local

Government Participation

The dress rehearsal locations include 62 jurisdictions—Sacramento,

Menominee County, and 60 jurisdictions (11 counties and 49 towns and

cities) in the 11-county South Carolina area. Sacramento and Columbia as

well as Menominee County and the state of South Carolina all were

involved with the LUCA program, although participation was far from

universal among the South Carolina jurisdictions. In South Carolina, the

Bureau’s Charlotte Regional Office officials (1) contacted the mayors of

the 49 municipalities and the chairs of the 11 county councils, (2) made

presentations to county councils and the City Manager for Columbia about

the importance of reviewing the census address list, and (3) mailed

numerous letters to the jurisdictions. However, despite the Bureau’s

efforts, it received responses to the LUCA review, in terms of annotated

maps and/or address changes, for only 34 of the 60 jurisdictions. South

Carolina’s Office of Research and Statistics, which maintains geodetic data

for accurate large-scale mapping, provided assistance to or conducted

12P.L. 103-430, Oct. 31, 1994.

Page 20 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

LUCA for an additional 10 of 60 the jurisdictions for a total participation

rate of 44 jurisdictions. Due to resource constraints, this office was unable

to assist two other jurisdictions.

According to Bureau regional officials, reasons jurisdictions gave for not

participating in the LUCA review included lack of resources and/or lack of

information to update address lists or maps. Our discussions with officials

from 5 of the 16 South Carolina area jurisdictions that did not participate

in LUCA and that responded to our inquiries, confirmed that the

jurisdictions’ primary reason for not participating was a lack of resources.

For example, an official from one South Carolina town said that the town

would have liked to participate, but that there was no one locally who

would agree to work on or help out with LUCA. Similarly, an official from

another South Carolina town said that it too would have liked to

participate. However, she noted that she would have been tasked to do the

job, and that there was insufficient time to perform her full-time job with

the town as well as review the Bureau’s address list and maps.

Our discussions with officials from 12 South Carolina area jurisdictions

that did participate in LUCA and that responded to our inquiries, indicated

that the potential cost of participating in LUCA can be high. For example, an

official in one South Carolina county estimated the cost to be $5,000, with

staff having to perform LUCA as well as their normally assigned duties in

the tax assessor’s office. An official from another city was unable to

provide a cost estimate, but indicated that the city’s Geographic

Information System specialist had spent about 4 weeks full-time on the

effort, and that several others had spent part of their time on the effort. An

official in another city said that the city committed 4-1/2 weeks to LUCA.

This official said that, during that time, two people worked full-time, 9

hours a day, driving down each street in the city. After that initial

canvassing, two people worked full-time and two other people worked

part-time for the remaining 2 weeks correcting the maps and address lists.

Sacramento officials estimated that city officials spent about 90 hours over

the course of 5 weeks, while the officials from the local area council of

governments spent about 450 hours. Even with these resources, these

officials estimated that only about half of the amount of work required for

LUCA was accomplished. This work included reconciling addresses in

blocks where the city and the area council of governments believed the

Bureau missed addresses and reviewing and correcting maps.

Page 21 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

Even with the resources committed to conducting reviews in those

jurisdictions that participated in LUCA, officials in the six jurisdictions we

contacted and that responded to our inquiries said that the amount of time

allowed for LUCA during the dress rehearsal—about 5 weeks—was

insufficient. According to Sacramento officials, time constraints resulted

in the city only being able to completely review 17, and partially review

another 15, of the 48 maps the Bureau provided the city; 16 maps were not

reviewed at all. Officials from one South Carolina jurisdiction said that

because of insufficient time, local officials did not focus any efforts on

reviewing addresses for possible deletions. Instead, they concentrated

their efforts on adding addresses even though they acknowledged that

these addresses may already have been located elsewhere in the address

files without their knowledge.

Limited Bureau Assistance

and Poor MAF and Map

Quality Hampered Local

Review

Those jurisdictions participating in LUCA reported several obstacles that

made their reviews more difficult. For example, officials in some

jurisdictions stated that the Bureau was able to provide little assistance to

them while they were reviewing the address lists and maps. Furthermore,

the accuracy, completeness, and format of the address lists and maps

hindered their review.

According to Bureau officials, although the Bureau’s Seattle and Charlotte

Regional Offices each designated a LUCA coordinator, the Seattle

coordinator for Sacramento was responsible for just a single jurisdiction.

In contrast, the Charlotte LUCA coordinator was responsible for all 60

jurisdictions in the South Carolina dress rehearsal site. This difference in

workload could be responsible for the different levels of satisfaction

between the two sites with the quality of the Bureau’s assistance, and

suggests that the Bureau’s workload in 2000 may significantly limit its

ability to provide needed assistance to localities.

For example, according to Sacramento officials, the Bureau’s Seattle

Regional Office staff was available at all times during the LUCA review, and

the level and quality of Bureau support was sufficient. However, a South

Carolina county official said that, while early contact with the Bureau was

open and forthcoming, during the LUCA review Bureau staff provided little

to no help. According to this official, when the county contacted the

Bureau about problems it was experiencing with the Bureau’s electronic

files and maps, it was told that the Bureau was aware of these problems

but that it could not provide any guidance or help. Similarly, officials from

one of the South Carolina cities said that the Bureau was of limited help in

Page 22 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

providing guidance and assistance during their review of the files and

maps. According to these officials, they were told that because of the

Bureau’s limited resources, local governments were relatively “on their

own” when conducting LUCA.

The Bureau’s assistance to local governments in 2000 is expected to be

significantly less than that provided to South Carolina during the dress

rehearsal. According to a Charlotte Regional Office official, while the

Bureau had one staff person available to assist the 60 local jurisdictions

during the dress rehearsal, this level of support will greatly diminish for

the 2000 Census. According to Bureau officials, on average, each region

will have about 11 partnership specialists for the 2000 Census, of which

about half are to focus primarily on LUCA. Therefore, the LUCA coordinators

in the Charlotte office would be responsible for about 2,200 communities

in that 5-state region, or about 200 to 400 communities each.

The accuracy, completeness, and format of the address lists and maps

were also an issue to participating jurisdictions, and an issue that could

reduce local governments’ ability to improve the accuracy of the MAF and

maps in 2000. According to Sacramento officials, the Bureau’s address file

for the city contained about 198,000 addresses, which were more

addresses than were contained in Sacramento’s database of addresses.

Sacramento officials found that the Bureau’s address file included

addresses outside the city limits, addresses not assigned to any block or

tract number, and commercial addresses.13 This significantly contributed

to problems in reviewing the Bureau’s address file. Sacramento officials

also said that the electronic address file provided by the Bureau did not

include information on where each address was located on the maps,

which created additional work for the city.

Similar concerns were voiced by South Carolina officials. These officials

said that the maps did not include street corrections, street additions, and

block adjustments made to the Bureau maps about 1-1/2 years earlier as

part of the Bureau’s Block Boundary Suggestion Program. Furthermore,

the maps did not include changes in city boundaries due to annexations

that are provided annually to the Bureau under the Bureau’s Boundary and

Annexation Survey. As a result, according to officials from the city of

Sacramento and the Sacramento Area Council of Governments, they

expended considerable time reviewing the LUCA maps to make corrections

that they had provided earlier to the Bureau. Similar problems were voiced

13A tract is a neighborhood of about 4,000 people and 1,700 housing units, designed to have

homogeneous population characteristics, economic status, and living conditions.

Page 23 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

by South Carolina officials. For example, according to officials in one

South Carolina county, the same errors were on the LUCA maps that the

county has been trying to correct since the 1990 Census.

According to Bureau officials, maps may contain errors for a variety of

reasons, including the Bureau’s inability to make corrections on some

Bureau maps when local governments provide corrections and revisions

on their own maps, as opposed to the Bureau’s maps. The Bureau cannot

make some of the local government’s corrections and revisions because

Bureau maps are not always spatially accurate and thus Bureau staff may

be unable to make the fine corrections necessary to accurately reflect

changes on locally produced, spatially correct maps.

The Bureau’s

Reengineering Plans

Address Some, but

Not All, Concerns

Cited by Local

Officials About LUCA

According to the Bureau, the reengineered approach to developing its

address list addresses several problems cited by jurisdictions that

participated in LUCA during the dress rehearsal. However, while the revised

approach does address some problems, it does not fully address them all.

Furthermore, the feasibility of the Bureau’s plan remains questionable.

To remedy the problem of lack of sufficient time to perform LUCA, the

Bureau plans to begin the LUCA process earlier and allow more time for

local review. Rather than have LUCA begin in late 1998 and continue until

the second quarter of 1999, the Bureau plans to begin LUCA in early 1998

and continue it through the second quarter of 1999.

To address the lack of participation in LUCA, the Bureau plans to establish

stronger relationships with state data centers, metropolitan and regional

planning organizations, and councils of government, especially in areas

where local governments are unable to participate. However, based on the

Bureau’s dress rehearsal experience, it is questionable whether such

organizations can provide the assistance envisioned by the Bureau. For

example, according to officials from the South Carolina Office of Research

and Statistics, the organization lacked sufficient resources to provide

assistance to all of the jurisdictions in the SC dress rehearsal site that

requested help. Should assistance be required on a statewide basis, these

officials said that the research and statistics office would not have

sufficient resources to provide it.

The Bureau’s reengineered approach does not address the quality of the

address lists or maps cited by officials from some of the dress rehearsal

jurisdictions. Although Bureau canvassing could improve the quality of the

Page 24 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 2 

The Quality of the Address List Has

Significant Implications for the 2000 Census

address lists and maps to a certain extent, because the Bureau’s

reengineered approach provides for an earlier LUCA, canvassing will not

occur in areas having city-style addresses until after local governments

have completed their review of addresses and maps. The Bureau’s

reengineered approach also does not address the level of Bureau

assistance available to local governments that choose to participate in

LUCA, even though it does call for providing more training. Further, given

the Bureau’s decision to reengineer its address list development efforts,

the Bureau will not be able to test its revised approach before

implementing it for the 2000 Census.

Page 25 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

To increase awareness of the 2000 Census and boost mail response rates,

the Bureau plans an extensive outreach and promotion campaign.

Effective outreach and promotion are essential because they have the

potential to boost the mail response rate, thereby reducing the Bureau’s

nonresponse follow-up workload and, thus, the cost of the census.

Through targeting, outreach and promotion can also help reduce any

undercount of traditionally hard-to-enumerate populations.

The Bureau expects that its outreach and promotion efforts, combined

with other initiatives, such as simplified census questionnaires, should

produce a mail response rate of 66.9 percent for the 2000 Census. This is

12 percentage points higher than the 55-percent response rate that the

Bureau expects it would achieve without these activities and slightly

higher than the 65-percent response rate achieved in the 1990 Census.

Nevertheless, the Bureau’s experience thus far during the dress rehearsal

suggests that, in 2000, this goal might be difficult to achieve.

The Bureau Has

Initiated Several

Outreach and

Promotion Activities

According to the Bureau, the success of its outreach and promotion efforts

will depend heavily on the effectiveness of the partnerships it hopes to

build with state, local, and tribal governments; nongovernmental

organizations; business; and the media. As a matter of long-standing

policy, the Bureau has maintained that it cannot seek to shoulder the

major financial burden of local census outreach and promotion efforts.

Thus, the Bureau is placing a priority on working with partners because

they can, among other things, help spread the word about the census,

encourage active participation, dispel myths, and issue public statements

of endorsement. Among the major activities we examined for this report

were the Bureau’s efforts to form Complete Count Committees (CCC),

implement its Census in Schools Project, and contract-out for a

paid-advertising campaign.

One of the primary goals of the CCCs is to heighten public awareness of the

census through community outreach activities. Bureau guidance

recommends that local governments form CCC’s consisting of elected

officials, business leaders, minority groups, religious organizations,

community-based organizations, and other individuals and groups. Such

partnerships could provide the Bureau with local knowledge, experience,

and expertise to help it take a more accurate census and ensure

productive use of the Bureau’s limited resources. For example, CCCs could

assist the Bureau by alerting Bureau officials to the existence of

hard-to-count population groups within their communities. These groups

Page 26 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

could then be targeted for more intensive and/or specific types of

advertising.

Bureau guidance provides potential CCCs with specific projects that they

could undertake to reach individuals in their communities. Such projects

could include working with businesses to (1) obtain their commitment to

display promotional flyers about the census, (2) insert reminders about the

census in customers’ billing statements or employees’ paychecks, or

(3) set up a census assistance table where volunteers would assist people

in completing their census forms. Other projects could involve obtaining

commitments from local radio stations or newspapers to provide media

coverage and publicity. The Sacramento and Menominee County CCCs, as

well as CCCs in some of the South Carolina counties, have initiated these

and/or similar activities.

Through its second partnership activity, the Census in Schools Project, the

Bureau hopes to increase response rates by using students to encourage

and, where necessary, help their parents or guardians complete census

forms. The schools project is also intended to (1) organize millions of

teachers, students, and families to support the census: (2) build a database

of interested teachers who may be recruited as census workers; and

(3) improve students’ understanding of the census process and why it is

important to the country. As currently planned, the schools project is to

include grades kindergarten through 12 as well as classes for adult English

as a second language, citizenship, and high school equivalency degrees.

The project also is to focus on five types of schools: public, private,

parochial, Bureau of Indian Affairs, and Department of Defense. College

and university students will be not be included in the schools project

because the Bureau believes that these students can be better reached

through traditional advertising.

A third partnership activity is the Bureau’s paid-advertising campaign.

According to the Bureau, the campaign is intended to increase mail

returns from the general public, targeted audiences, and historically

undercounted populations. In the 1990 Census, the Bureau relied on pro

bono public service advertising to get its message across. However, such

advertising had its limitations. For example, the Bureau has noted that

television announcements were often aired at non-prime-time hours when

audiences were relatively small. For the 2000 Census, the Bureau

concluded that a paid-advertising campaign would allow it to better

choose where and when to place advertisements, more effectively reach

hard-to-count populations, and present a more uniform message.

Page 27 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

On October 10, 1997, the Bureau announced that it had awarded its 2000

Census paid-advertising contract to Young & Rubicam (Y&R), which is a

private advertising agency. A Y&R subcontractor, Scholastic, Inc., is to

develop a package of educational materials and a marketing plan for the

Bureau’s schools project.

According to Bureau officials, the Bureau has budgeted about $230 million

for 2000 Census outreach and promotion activities. Of that amount, about

$100 million is for the Y&R paid-advertising campaign. Of the $100 million,

about 80 percent has been earmarked for buying advertising in print and

the broadcast media. The remaining 20 percent is to be used for such

activities as contract administration and overhead. Of that 20 percent, a

portion is to be used for an independent evaluation of the effectiveness of

the paid-advertising campaign.

Another $100 million of the $230 million outreach and promotion budget is

to be used to hire as many as 320 full-time equivalent (FTE) partnership

specialists, including government, community, and media specialists.14 The

remaining $30 million is to fund (1) per diem and travel expenses for

members of census advisory committees, (2) the Census in Schools

Project, and (3) certain local promotional activities, such as printing.

Effectively

Implementing the

Bureau’s Outreach

and Promotion

Activities Could Be

Problematic

Although the Bureau’s plan to publicize the 2000 Census was designed to

address the outreach and promotion problems encountered during the

1990 Census, the results to date of the dress rehearsal suggest that several

obstacles may diminish its effectiveness. These obstacles include

inconsistent levels of participation by local communities, a lack of

resources at both the Bureau and local level, and resistance to the

Bureau’s message that the census is important. As a result, the Bureau may

have difficulty achieving its goal of a 66.9 percent mail response rate in

2000.

14An FTE or work year generally includes 260 compensable days or 2,080 hours. According to the

Bureau, government specialists will be responsible for working with local governments to review local

address files; establish CCCs; and obtain space to recruit and train census workers, among other

activities. Community specialists are to work with local groups to raise awareness of the census, help

recruit census workers, and obtain testing space. Media specialists are to generate publicity about the

census by working with print and electronic media.

Page 28 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

Participation in Outreach

and Promotion Activities at

the Dress Rehearsal Sites

Has Been Inconsistent

Although the Sacramento and Menominee County CCCs appear to be

active, efforts to establish CCCs in South Carolina have had mixed results.

Of the 11 counties and City of Columbia included in the South Carolina

dress rehearsal, just 3 counties and Columbia had active CCCs at the time

of our review. The remaining eight counties either had not formed CCCs or

those that were formed had become inactive.

Reasons for the nonfunctioning CCCs in South Carolina varied and were at

times beyond the Bureau’s control. For example, according to a Bureau

official, one county’s CCC disbanded after a key member left. In at least two

instances, however, it appears that the Bureau was at least partly

responsible for the operational problems that caused the CCCs to suspend

their activities. For example, one local official said that CCC members

became frustrated and dissatisfied with the lack of organization and

leadership from representatives of the Bureau’s regional office. An official

from another CCC said that vague communications between the Bureau

and the CCC played a role in the CCC’s cessation of activities. The official

said the Bureau did not set clear expectations for what the CCC was

supposed to do, and that the Bureau had supplied only minimal guidance

and support.

Communication difficulties appeared to be a problem with several active

South Carolina CCCs as well. Four of the six active CCCs we contacted at

the three dress rehearsal sites indicated that the Bureau did not set clear

expectations for their CCCs, especially when they were first initiated,

and/or Bureau guidance and literature has been minimal.

Local and Bureau

Resources Affected Results

to Date of Outreach and

Promotion Efforts at the

Dress Rehearsal Sites

Local and Bureau resources may not be adequate for effective outreach

and promotion. While the Bureau generally plans to provide communities

with in-kind services (e.g., copies of flyers), local communities are to

provide critical funding during the 2000 Census. In practice, however, this

arrangement has led to some operational problems with the CCCs at the

dress rehearsal locations. For example, officials from five South Carolina

counties and the City of Columbia indicated that resources to carry out

promotional activities were limited. In Sacramento, a city official said that

much of what the city’s CCC proposed to do in the way of media advertising

will depend on the funding available from the Bureau’s advertising

contractor and any in-kind services provided by the Bureau.

Moreover, although local officials said that some local businesses have

agreed to undertake such activities as displaying promotional flyers, as a

Page 29 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

rule, local businesses have not provided and likely will not provide funding

for promoting the dress rehearsal. For example, in South Carolina,

officials from five counties said that funding and in-kind services from

local businesses would probably not be available. The Bureau’s

partnership specialist for the South Carolina site acknowledged that local

business support has been sporadic and that business partnerships the

Bureau had hoped for have not yet occurred. In Sacramento, a local

official noted that while businesses would likely be enthusiastic about

certain in-kind services, such as displaying posters and including flyers in

mailings, they probably would not provide any funding. At the Menominee

County dress rehearsal site, a tribal official noted that the reservation

contained few businesses, virtually all of the businesses were small, and

none had the resources to contribute to promoting the census.

The costs of supporting CCC activities typically were absorbed by local

governments. An official from one South Carolina county estimated that

the county’s CCC has spent 100 staff hours on CCC initiatives, while an

official on the City of Columbia’s CCC said its CCC members have spent

about 240 staff hours on outreach and promotion activities as of early

December 1997 and will probably spend an equal amount before the dress

rehearsal is finished. An official from another South Carolina county

estimated that its CCC has spent $2,000 in cash and in-kind services for

outreach and promotion efforts and that it will spend about $5,500 more

before the dress rehearsal is completed. An official from another South

Carolina CCC told us that outreach and promotion activities have cost

about $8,000 to date, and the official anticipated that the CCC would spend

an additional $27,000 before the dress rehearsal is over.

As previously noted, the Bureau ultimately plans to fill as many as 320 FTE

partnership specialist positions for the decennial census in its 12 regional

offices. Given the approximately 3,100 counties and county equivalents

that comprise the nation, each full-time partnership specialist’s caseload

would average 10 counties. This is only slightly less than the workload of

the Bureau’s South Carolina partnership specialist, who, as previously

described, was responsible for mobilizing outreach and promotion efforts

in 11 counties and the City of Columbia and achieved only mixed results.

(The South Carolina partnership specialist’s workload did not go

unnoticed; several local officials from five of the South Carolina dress

rehearsal sites observed that the partnership specialist might have been

stretched too thin.) Furthermore, in addition to outreach and promotion

efforts during the 2000 Census, the partnership specialists will be

responsible for additional activities, such as responding to media inquiries,

Page 30 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

obtaining space for recruiting and training census workers, and assisting

with LUCA reviews.

Local resources for promoting the census was an issue during the 1995

Census Test. As we noted in a previous report, the 1995 Census Test

indicated that cooperative ventures with local governments provided a

way to promote public participation in the census. However, as we

reported, local governments in urban areas where response rates were

lowest reported that their lack of funding to promote the census was

problematic.15

The Bureau is also implementing the Census in Schools Project at the

dress rehearsal sites and, as with the CCCs, is encountering inconsistent

participation. In South Carolina, the Bureau’s partnership specialist told us

that six counties were participating in the program at the time of our

study. As was the case with the South Carolina CCCs, county officials

involved with the schools project cited problems such as unclear

expectations from the Bureau and/or lack of resources as the reasons for

their difficulties.

The Bureau’s

Paid-Advertising Campaign

Faces Challenges

The Bureau’s national advertising contractor, Y&R, began its efforts for the

dress rehearsal in mid-December 1997 by convening a series of focus

group interviews with different segments of the population at the dress

rehearsal sites. According to Bureau officials, the purpose of these focus

groups was to build on earlier, more extensive research that was

conducted by the Bureau on personal reactions to the census, and to test

proposed advertising themes and approaches that could be used to reach

specific, targeted audiences during the 2000 Census.

Currently, the advertising campaign is based on the theme “This is your

future—don’t leave it [blank]” and stresses how responding to the census

questionnaire benefits one’s community. This same general theme was

also used for the 1990 Census. However, Bureau officials are hoping that

the paid-advertising strategy will allow more effective ad placement as

well as more precise targeting of specific messages to particular groups

compared to the pro bono promotional services that were used in previous

censuses.

The focus group results give an indication of the task Y&R faces in

developing an effective campaign. While some participants in the sessions

15GAO/GGD-97-142.

Page 31 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 3 

The Bureau’s Mail Response Rate Objective

Could Be Difficult to Achieve Because of

Obstacles to Implementing Its Outreach and

Promotion Strategy

we observed were generally aware of the census and its purpose, others

were unfamiliar with the census. Furthermore, while some participants

reacted positively to the test advertisements they were shown, others were

more cynical and remained unconvinced that they should respond to the

census. Cultural and other demographic differences will likely reinforce

the difficulties confronting the Bureau. Moreover, as demonstrated by the

results of the 1990 Census, simply raising public awareness is insufficient.

The public must also be motivated to return their questionnaires in the

face of a long-term decline in the mail response rate. The Bureau’s

research found that, although about 93 percent of the public was aware of

the 1990 Census, the mail response rate was only 65 percent. This was

10 percentage points lower than the mail response rate in the 1980 Census.

Page 32 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 4 

The Bureau’s Ability to Staff the 2000

Census Is Unclear

Recruiting, hiring, training, and retaining a workforce sufficient to

accomplish the decennial census is undoubtedly one of the government’s

greatest human resource challenges. Although the Bureau has a number of

initiatives aimed at making it easier to recruit and retain census workers, a

number of uncertainties surround the Bureau’s efforts and their likely

success in 2000.

The Bureau Has

Developed Several

Initiatives to Address

Potential Staffing

Difficulties but Could

Still Encounter

Problems in 2000

The Bureau estimates that, under its current design, it will need to fill

about 295,000 office and field positions to carry out various activities for

the 2000 Census. Key positions include census takers (enumerators), crew

leaders, field operations supervisors (supervisory enumerators), and

clerical office workers. To fill this many positions, the Bureau estimates it

will need to recruit some 2,632,600 applicants because, for a variety of

reasons, most applicants never make it through the employment process.

Some eventually take other jobs, others do not report for testing or

training, while still others fail background checks and employment tests.

Overall, the Bureau says that for every 10 applicants, only 1 is ultimately

hired.

Aside from the sheer number of applicants needed, staffing the census is

difficult because most census jobs are temporary and do not offer such

benefits as health insurance, life insurance, sick or annual leave,

retirement plans, or child care. However, census workers are eligible for

unemployment and worker compensation. Also, the majority of the office

and field positions must be filled during the short window when

nonresponse follow-up operations are conducted. As currently planned,

for the 2000 Census, this window will occur during a 6-week period

beginning the end of April 2000 and lasting through early June. During that

time, the Bureau has set a staffing goal of 121,000 people.

As we reported in our previous work, during the 1990 Census, the Bureau

encountered a number of staffing problems that adversely affected both

the cost and quality of the census. For example, almost one-half of the

Bureau’s district offices were unable to meet the 1989 recruiting goal of

four applicants for every field position for precanvassing operations,

during which census workers verified the completeness and accuracy of

address lists. As a result, 14 percent of the 109 district offices finished the

operation at least a week behind schedule.16 Furthermore, three of the

Bureau’s regional offices reported that over 70 percent of their district

161990 Census: Enhanced Oversight Should Strengthen Recruitment Program (GAO/GGD-90-65, Apr.

13, 1990).

Page 33 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 4 

The Bureau’s Ability to Staff the 2000

Census Is Unclear

offices were understaffed as of early May 1990—a peak period for

following up on nonresponding households.17

The Bureau Plans Several

Initiatives to Ensure It Will

Have an Adequate

Workforce, but Staffing the

2000 Census Could Still Be

Difficult

To address the potential problem of recruiting, training, and retaining

almost 300,000 staff to fill office and field positions, the Bureau plans to

use several recruitment and compensation strategies. For example, during

the 1990 Census, the Bureau focused its recruitment efforts on

unemployed individuals; however, during the 2000 Census, the Bureau

plans to target its hiring toward individuals seeking “part-time, part-time”

employment (i.e., individuals who are already employed but who are

seeking additional work (moonlighters)). Retirees wishing to supplement

their income are also a focus of the Bureau’s hiring efforts.

Bureau officials said that moonlighters and retirees are more likely to find

the temporary, intermittent weekend and evening work of an enumerator

attractive. The officials also said such individuals may be more likely than

unemployed individuals to have existing health insurance and other

benefits as well as the needed job skills to do the work. Furthermore, they

might be in a better position to work evenings and weekends when more

people are likely to be home, thus making the Bureau’s nonresponse

follow-up efforts more efficient.

To further reduce recruitment and retention problems such as those

experienced in the 1990 Census, the Bureau plans to implement a

wage-setting program, key features of which include locality-based pay

and bonuses for exceeding production. The Bureau’s compensation

program is based on recommendations contained in a June 1997 interim

report from Westat, which is a private consulting firm. The Bureau asked

Westat to analyze factors (pay, working conditions, demographic and

economic trends) that appeared to have contributed to the success or

failure of hiring that took place in the 1990 Census. The Bureau plans to

set the wages it pays census workers at 60 to 80 percent of the prevailing

wage rate for the local area as reported by the Bureau of Labor Statistics.

According to Bureau projections, the resulting pay rates would range from

$8 to $15 per hour.

The Bureau also plans to pay Census employees bonuses for exceeding

production and quality standards and for completing assigned work for

selected operations. For example, enumerators who obtain completed

17Progress of the 1990 Decennial Census: Some Causes for Concern (GAO/T-GGD-90-44, May 21, 1990).

Page 34 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 4 

The Bureau’s Ability to Staff the 2000

Census Is Unclear

census questionnaires at the rate of 1.5 cases per hour during nonresponse

follow-up operations are to receive a bonus of about $2.00 per case.

To expand the potential applicant pool, the Bureau has taken steps to

make it easier to hire federal employees and individuals on public

assistance without affecting their pay and benefits. For example, the

Bureau has the authority to waive pay/retirement reduction requirements

for federal military and civilian retirees who work on the 2000 Census. The

Bureau is also seeking assistance from the Office of Personnel

Management to remove certain administrative barriers to make it easier to

hire current federal employees to work on the census. For individuals on

public assistance, for example, the Bureau requested the Department of

Housing and Urban Development to exclude the amounts earned by

census enumerators when calculating annual income for eligibility in

public and assisted housing programs. According to the Bureau, the

Department approved the request in May 1996.

Despite these initiatives, the Bureau may find that its efforts to adequately

staff the 2000 Census could be problematic. On top of the hiring

difficulties that the Bureau has encountered during past censuses, if

current national trends continue, the Bureau could be facing an extremely

tight labor market. In 1990, the overall unemployment rate was

5.6 percent, compared to 4.9 percent in 1997—the lowest since 1973.18

Furthermore, if the Bureau is prevented from using statistical sampling

and estimation procedures, recruiting and retaining a sufficient workforce

for the 2000 Census could be even more difficult. According to the Bureau,

if all nonresponding households are followed up by enumerators, an

additional 59,000 enumerators would be needed on top of the estimated

300,000 staff. Additional effort would also be required to verify whether

households were vacant. If the Bureau is allowed to sample, the Bureau

plans to verify 10-percent of the housing units identified as vacant by the

Postal Service. According to the Bureau, verifying a 10-percent sample of

vacant housing units would require 5,000 enumerators. However, in the

absence of sampling for vacant housing units, the Bureau estimates that

25,000 to 30,000 enumerators would be required for this operation. These

enumerators are in addition to those required for nonresponse follow-up.

18Historical unemployment rates and the estimated unemployment rate for 1997 are published by the

Bureau of Labor Statistics and contained in monthly reports entitled Current Population Survey Data,

Civilian Labor Force, Age 16 and Older.

Page 35 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 4 

The Bureau’s Ability to Staff the 2000

Census Is Unclear

The Bureau’s Recruitment

Strategy Was Based on

Limited Data

We are also concerned that the Bureau’s decision to focus its recruitment

efforts on moonlighters and retirees was based on limited data. Bureau

officials told us that the rationale for its decision was based, in part, on the

Bureau’s experience during the 1995 Census Test, when informal briefings

were held with enumerators. The officials noted that the Bureau did not do

a formal evaluation of the composition of the workforce for the 1995

Census Test.

The Bureau noted that its recruitment strategy decision was also based on

research conducted by Westat. However, Westat’s study acknowledges the

limitations of the data, which were partly based on Westat’s own hiring

practices and those of similar firms. Westat noted that most of its

recruiting, training, and surveying is conducted in the evening and on

weekends using employees that have other jobs or family responsibilities

during regular business hours.

However, we believe that Westat’s implicit assumptions that its workforce

(1) consists of similar kinds of employees who would be interested in

census work and (2) is comparable nationwide, in the absence of any

other data, may or may not be valid. For example, as noted by a Westat

official, unlike census enumerators who are employed in all parts of the

country to collect information by going door-to-door, the majority of

Westat’s interviewers work in the Washington, D.C., area, and conduct

interviews by telephone. Further, although Westat conducted a detailed

analysis of the nonresponse follow-up workforce used during the 1990

Census in Pennsylvania, it lacked certain data relevant to its

recommended recruitment strategy. For example, the resulting report

recommended further analysis to determine such key relationships as how

well prior employment and earnings predict recruitment and retention and

how productivity varies as a function of employment patterns.

The Dress Rehearsal Will

Not Validate the Bureau’s

Staffing Initiatives

According to Bureau officials, staffing the dress rehearsal activities that

have occurred to date has gone well. The Bureau was able to hire enough

people to adequately staff its operations. Nevertheless, the demand for

temporary census workers thus far has been small, and more generally,

the Bureau’s experience gained during the dress rehearsal does not and

cannot provide an adequate test of the effectiveness of the Bureau’s

staffing strategy for the 2000 Census.

According to officials in the Bureau’s Seattle Regional Office, the Bureau

has been able to adequately staff its operations for the Sacramento dress

Page 36 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 4 

The Bureau’s Ability to Staff the 2000

Census Is Unclear

rehearsal. They noted that the quality of the people hired, to date, has been

good, and has included people with master’s degrees and retired corporate

executives. Similarly, officials in the Bureau’s Charlotte Regional Office

stated that they have had no difficulty attracting and hiring the necessary

staff.

Nevertheless, only a relatively small amount of hiring has taken place. As

of January 1998, Bureau workforce data indicate that, for calendar year

1997, a total of only 488 individuals were hired at the three dress rehearsal

sites. Of the 488 individuals hired, 352, or about 72 percent, worked at

various locations at the Columbia site, primarily conducting address listing

operations. Sacramento dress rehearsal operations accounted for 127 of

the 488 hires. These individuals were involved with address development

operations. Very limited hiring was done in calendar year 1997 to staff

dress rehearsal operations at the Menominee County site, where nine

individuals were hired to complete address listing activities at this

location. Many more individuals will be needed after Dress Rehearsal

Census Day for nonresponse follow-up operations and statistical sampling

procedures in Sacramento.

However, the dress rehearsal is not a good test of the Bureau’s staffing

strategy for the 2000 Census. First, the labor market at the dress rehearsal

sites is not necessarily representative of what the Bureau will encounter

nationally in the 2000 Census. For example, according to Bureau officials,

recruiting at the Sacramento site was made somewhat easier by the

closure of a nearby air force base, which added more people to the

potential applicant pool.

Moreover, as the demand for census workers increases during the dress

rehearsal for such operations as nonresponse follow-up, the Bureau can

compensate for any local hiring difficulties by shifting employees to where

they are needed most, or even recruiting outside of the dress rehearsal

jurisdictions. In the 2000 Census, with each community being called upon

to supply the necessary resources to carry out census activities, the

Bureau will be less able to use such measures.

Page 37 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 5 

The Bureau’s Plans for Sampling and

Statistical Estimation Face a Number of

Challenges

Declining response rates have posed problems for the Bureau since it

began its mail-out/mail-back procedure in 1970. Hundreds of thousands of

additional enumerators must be hired to collect census information from

an increasing number of nonresponding households. To reduce its

nonresponse follow-up workload, the Bureau plans to sample

nonresponding households for the 2000 Census. The Bureau has also

designed a procedure called Integrated Coverage Measurement (ICM),

which involves taking a separate sample after the nonresponse follow-up

is completed to make adjustments to the census counts. However, these

activities face several methodological, technological, and quality control

challenges. The 1998 dress rehearsal may provide experience useful to

addressing these challenges, but others may surface as time goes on.

The Bureau Plans to

Use Statistical

Methods to Reduce Its

Nonresponse

Follow-Up Workload

In 1990, the Bureau sent Census questionnaires to most households in the

United States, asking that household members complete the questionnaire

and mail it back to the Census Bureau. The Bureau hired enumerators to

visit and collect data from the 35 percent of households that did not return

a questionnaire. This follow-up effort proved costly. About $560.3 million

of the total $2.6 billion allocated for the 1990 Census (about 50 percent of

the data collection budget) was spent on nonresponse follow-up.

Furthermore, the effort ran substantially behind schedule because a

number of housing units were particularly difficult to contact and required

repeated attempts to enumerate.

Because of the difficulty in enumerating these housing units, on occasion,

the Bureau had to rely on “last resort” and close-out data collected from

neighbors or other knowledgeable sources for those housing units that

could not be personally contacted.19 Such data has been shown in the past

to be less accurate than data collected directly from housing unit members

because external sources cannot be expected to have complete

information about a housing unit or to know if its members were counted

elsewhere. For example, in our 1992 report, we found that these

procedures may have contributed to the decline in the quality of 1990

Census data because the Bureau accepted less complete responses and

information from nonhousehold respondents.20 Bureau data from the 1990

Census show that last resort and close-out procedures were used to gather

19Last resort data are collected either from someone outside of the household, such as a postal carrier,

neighbor, landlord, or relative, or from a household member who does not wish to provide complete

information. Closeout data are collected when last resort information is unavailable and includes

minimal information, such as whether a unit is occupied or nonexistent.

20GAO/GGD-92-94.

Page 38 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 5 

The Bureau’s Plans for Sampling and

Statistical Estimation Face a Number of

Challenges

information on 6.6 percent of the persons enumerated during nonresponse

follow-up. We reported that these procedures were used most heavily in

urban areas and among minority populations. Last resort and close-out

data accounted for an average of 20 percent of the households in large-city

census offices and 4.9 percent of the non-Hispanic Black enumerations

(compared to 2.3 percent of the non-Hispanic White enumerations).

For the 2000 Census, the Bureau anticipates that the mail response rate

will be 66.9 percent, which is essentially the same as the 65 percent mail

response rate achieved in the 1990 Census.21 Rather than following up with

visits to every nonresponding housing unit, the Bureau has proposed a

strategy to enumerate remaining housing units on the basis of a sample of

nonresponding housing units in each census tract. After the end of the

mail response phase, the Bureau will select a sample that would result in a

response rate of at least 90 percent of the housing units in each tract.

Sample data will then be used to impute the characteristics of the

remaining 10 percent of the housing units. To ensure equitable treatment

of census tracts with high response rates, all tracts with an initial response

rate of at least 85 percent in the mail-back phase will be sampled at a rate

of 1 in 3, which will result in more than 90 percent of the housing units in

these tracts being enumerated.22 The Bureau believes its plan will reduce

costs and the time necessary to conduct follow-up of nonresponding

housing units.

For the dress rehearsal in Sacramento, 2000 Census procedures will be

used for nonresponse follow-up. A systematic sample, sorted by address,

will be selected in each census tract once the mail-back phase of

enumeration is completed. If a census form is mailed back after the

nonresponse follow-up sample has been selected, the data from the mailed

form will be used for that household, and forms sent in by households will

continue to be accepted until the end of nonresponse follow-up data

collection.

21The Census Bureau projects that mailout of a single Census form—the procedure used beginning in

1970—would result in a mailback rate of 55 percent in 2000.

22As discussed in the following section of this report, enumerators will attempt to contact all housing

units in blocks selected for the Integrated Coverage Measurement Survey. Sample-based nonresponse

follow-up will not be utilized in these blocks.

Page 39 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 5 

The Bureau’s Plans for Sampling and

Statistical Estimation Face a Number of

Challenges

The Bureau Plans to

Adjust for

Enumeration Errors

Since the 1940 Census, the Bureau has conducted a variety of studies to

measure inaccuracies in census population counts. Methodologies have

varied over the years—Post-Enumeration Surveys (PES); demographic

analyses of official records such as birth and death certificates; and

matches to the Current Population Survey. Regardless of the method used,

a net undercount has been measured for each census since 1940. During

the 1990 Census, the Bureau conducted a PES of 167,046 housing units to

measure inaccuracies in the population count. However, then-Secretary of

Commerce Robert Mosbacher ruled that evidence for adjusting census

data, on the basis of the 1990 PES design, was inconclusive and

unconvincing. Consequently, the Bureau used PES results only to evaluate

the quality of census data, not to revise its final 1990 census numbers. To

measure inaccuracies in the population count for the 2000 Census, the

Bureau has proposed the ICM, which corrects several problems identified

in the 1990 PES and which relies on a much larger sample. The Bureau

expects to use the ICM results to produce a “one-number census.”

Based on the PES, the net national undercount (the number of people

omitted minus the number overcounted) in 1990 is estimated to have been

4 million people, or 1.6 percent of the population. The net undercount

reflects 10 million people who were not counted or who were omitted

from the census in their proper place of residence (i.e., omissions) and

6 million people who were either enumerated more than once, were

ineligible to be counted, or were counted at a locations other than their

correct place of residence (i.e., erroneous enumerations). While the net

undercount in 1990 was 4 million people, the differential undercount of

minorities and other hard-to-enumerate groups and areas was more

problematic. For example, in the 1990 census, the rate of undercount was

over four times higher for African Americans than for non-African

Americans. Evaluations of the 1990 Census have indicated that attempts to

add additional counting activities did not overcome this problem.

The ICM is designed to improve the accuracy of the population counts for a

variety of segments of the population and would be the last phase in

census data collection, following receipt of mail-out questionnaires and

participation in nonresponse follow-up. The Bureau is to draw a sample of

750,000 housing units throughout the United States for the ICM, based on

an address list created independently from that prepared for the census

enumeration. Enumerators are then to be sent to these housing units to

interview residents and obtain the characteristics of individuals living

there. In blocks selected for the ICM, sample-based, nonresponse follow-up

will not be used. Both the census enumeration and the ICM will attempt to

Page 40 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 5 

The Bureau’s Plans for Sampling and

Statistical Estimation Face a Number of

Challenges

collect information from all housing units in blocks selected for ICM. The

Bureau will then compare data collected by the ICM and by the Census

enumeration in ICM blocks to determine the degree to which individuals

enumerated by one data collection were also detected by the other.

Information from this process is to be used to estimate the extent to which

people were correctly enumerated, missed, or counted in error for the

entire census. The Bureau would estimate the correct population for entire

geographic areas, as well as for specific subpopulations as defined by

characteristics such as age, sex, tenure, race, and ethnic origin.

Sampling and

Statistical Estimation

Designs Face

Methodological,

Technological, and

Quality Control

Challenges

In our July 1997 report, we noted that sampling for nonresponse follow-up

could reduce costs and save time, while the ICM could improve the

accuracy of the population totals.23 Nevertheless, the Bureau faces a

number of operational challenges in implementing its sample-based, data

collection activities.

First, as currently designed, the schedules for completing certain phases of

the census—including mail response, nonresponse follow-up, and ICM—are

compressed when compared to the 1990 Census. In 1990, the Bureau

allowed 4 weeks from Census Day for mail response before beginning

nonresponse follow-up. In 2000, the Bureau also plans to allow 4 weeks

from Census Day for mail response. In 1990, nonresponse follow-up was

scheduled to last 6 weeks but in some locations lasted 14 weeks.24 For the

2000 Census, the Bureau will again allow 6 weeks for completion of

nonresponse follow-up. In 1990, PES data collection and reconciliation

were not completed until January 4, 1991, while for the 2000 Census, the

Bureau plans to perform the same tasks for 5 times the number of

households by the end of September 2000. Furthermore, ICM activities from

data collection to completion of ICM follow-up interviewing and matching

are scheduled to last 5 months. The PES performed after the 1990 census

used a much smaller sample of 167,046 housing units, yet it required more

than 6 months to complete.

Second, given these time constraints, the Bureau may have to limit

repeated call-backs to hard-to-enumerate housing units and rely more

heavily on other sources for last-resort or closeout census data to

complete the nonresponse follow-up phase on time. In the 1990 Census,

the Bureau’s reliance on last-resort data varied by location, with large city

23GAO/GGD-97-142.

24In 1990, Bureau enumerators were able to complete about 90 percent of their follow-up workload

within 8 weeks, with the final 10 percent requiring more than 6 weeks to complete.

Page 41 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 5 

The Bureau’s Plans for Sampling and

Statistical Estimation Face a Number of

Challenges

census offices enumerating an average of 20 percent of all nonresponse

follow-up households by last-resort or closeout procedures. The quality of

census enumeration data can suffer when the Bureau uses last-resort or

closeout data, because other sources, such as landlords, neighbors, or

relatives, may not have complete information on the occupants of a

housing unit.

Third, the quality of the final census numbers, as adjusted by the ICM, will

be highly dependent on accurate and precise ICM measurement of those

persons who were properly enumerated and those who were omitted. The

Bureau has set a target of a 98 percent completion rate for ICM

interviews—which was the rate it was able to achieve for PES during the

1990 Census—to provide the data quality it expects for adjusting the

population count.

As noted earlier, completion schedules for ICM activities are compressed

when compared to 1990 PES time frames. A potential consequence of

failure to complete ICM on schedule is that the completion rate for ICM data

may fall below the expected 98 percent, which could adversely affect the

quality of ICM data and therefore the final adjusted census counts.

More specifically, a lower ICM completion rate may result in some

segments of the population being underrepresented in ICM data.

Comparisons of housing units interviewed during the 1990 PES indicate

that it was more difficult to complete interviews in housing units

composed of singles, minorities, and renters. Thus, if the ICM completion

rate is below the 98 percent target, there is increased risk that these

hard-to-count groups will be underrepresented in ICM data, and that

ICM-based adjustments may not reflect different segments of the

population accurately. Reaching the targeted completion rate for

hard-to-enumerate populations may require the Bureau to rely on a larger

proportion of last resort and close-out data, which, as noted earlier, are

typically of lower quality.

Fourth, difficulties in properly identifying and matching ICM information

back to data submitted by ICM households during the enumeration phase of

the census may affect the usefulness of ICM data files and estimates. In

December 1997, the U.S. Department of Commerce Inspector General

reported that software designed to match an ICM sample housing unit to

the same census enumeration housing unit will only search within the

Page 42 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 5 

The Bureau’s Plans for Sampling and

Statistical Estimation Face a Number of

Challenges

block to which that housing unit has been assigned.25 A discrepancy of as

little as one digit in the housing unit address between ICM and the census

enumeration could result in the housing unit being located in different

blocks by ICM and census enumeration, thus precluding a match. To

improve the quality of its software development, the Bureau says it is

implementing the Inspector General’s recommendations, taking such steps

as using peer review and checklists to ensure the proper implementation

of sampling and estimation methodology in software.

Housing units selected for ICM whose residents move between Census Day

and the beginning of ICM data collection may create additional problems.

The failure to locate and interview the same individuals who lived at ICM

addresses at the time of the enumeration will lower the ICM completion

rate. Tracking individuals who move is important because such individuals

may differ in important characteristics, such as family size or marital

status, from less mobile individuals. As a fallback, data may be collected

from successor households in the housing unit or from other

knowledgeable sources outside of the housing unit, but these sources

cannot be expected to always have full or accurate information on

members of the original household.

Fifth, the Bureau’s implementation of the ICM for the Dress Rehearsal has

suffered from a number of quality control problems. During the course of

our work, we observed the Bureau’s difficulties in drawing a useable

sample of housing units to conduct the ICM for the dress rehearsal. The

Bureau initially drew an ICM sample from a list that included commercial

addresses that should have been excluded. Inclusion of these commercial

addresses rendered the initial sample unusable. The Bureau drew a second

ICM sample from a list that excluded these ineligible commercial

addresses. However, the Bureau’s specifications for selecting blocks with

different racial, ethnic, and renter/owner characteristics for the sample

contained a number of inconsistencies. As a result, there were several

errors in the number of housing units selected for the different categories

of blocks. Because these errors were not detected until relatively late in

the sample selection process, we are concerned about the Bureau’s ability

to detect and correct problems in a timely manner.

252000 Decennial Census: Key Milestones and Associated Risks, report by the U.S. Department of

Commerce Inspector General to Sen. John McCain, Chairman, Committee on Commerce, Science, and

Transportation, U.S. Senate, Dec. 30, 1997.

Page 43 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 6 

Conclusions

We believe that, although the Bureau has made progress in addressing

some of the problems that occurred during the 1990 Census, key activities

continue to face developmental and/or implementation challenges. We

recognize that difficulties are to be expected with such a massive and

complex undertaking as the census. Nevertheless, with the Decennial

Census Day just 2 years away, we find it troubling that the Bureau finds

itself facing several ongoing and newly emerging operational challenges.

In addition to these operational issues, the continued lack of an agreement

between Congress and the administration over the final design of the 2000

Census contributes to the uncertainty surrounding the Bureau’s ability to

conduct an accurate and cost-effective census. The risk of a failed census

in 2000 will increase as long as the impasse over its final design persists.

The dress rehearsal is the Bureau’s final opportunity for a large-scale,

operational test of its overall design. If properly planned, executed, and

evaluated, the dress rehearsal could provide the Bureau with valuable

information on the feasibility of its current approach and on where

corrective action might be needed. With little time remaining until the 2000

Census, a window of opportunity will open only briefly to allow the

Bureau to assess the results of the dress rehearsal and take the necessary

steps to conduct an accurate, cost-effective census in 2000.

Agency Comments

and Our Evaluation

The Acting Director of the Bureau of the Census provided written

comments on a draft of this report. These comments, along with our

response, are summarized below and printed in full in appendix I.

The Bureau agreed that the key census-taking activities we reviewed were

facing operational challenges, but expressed confidence that these

activities are well under control. The Bureau is concerned that delays and

uncertainties caused by the lack of agreement on the Bureau’s planned use

of statistical methods have increased the risks to the census. We believe

that the successful implementation of key census operations is a

prerequisite for a high quality census, regardless of whether or not the

Bureau uses statistical methods.

The Bureau also commented on our characterization of specific

operational issues. The Bureau generally agreed with the facts as

presented with regard to the original design for the address list

development methodology and the reengineered design. It also concurred

with our assessment that achieving a 99 percent inclusion rate for its

address list is one of the greatest challenges for the 2000 Census. However,

Page 44 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 6 

Conclusions

the Bureau disagreed with our statement that the reengineered design has

not been tested. It noted that the methodology is proven and has been

used in previous censuses and tests. We agree that the components of the

reengineered methodology—canvassing census blocks, working with the

Postal Service, and LUCA—have been used and tested, but the Bureau has

not used or tested them in concert with each other, nor in the sequence

presently designed for use in the 2000 Census. For example, in 1990, block

canvassing was done before local government became involved. Further,

local governments were not provided address lists, but instead just

numbers of addresses in particular census blocks. Moreover, data from the

1990 Census suggests that address list development methodologies may

not necessarily be uniform across all jurisdictions or sufficient to meet the

Bureau’s goal for the accuracy of its address list.

The Bureau also noted that, as part of its reengineering effort, it has taken

steps to improve LUCA, which it agreed did not progress smoothly during

the dress rehearsal. These steps include providing additional time for local

review, simplifying the process, and suggesting alternative strategies

requiring less intensive efforts for jurisdictions having limited resources.

However, as noted in the report, we believe that the initiatives will address

some, but not all, of the concerns cited by local officials. For example, we

remain concerned that the quality of the address lists and maps provided

to local governments may limit the effectiveness of local review.

The Bureau agreed with the importance of outreach and promotion

activities, but disagreed with our characterization that “results to date of

the dress rehearsal [with local outreach] suggest . . . the Bureau may have

difficulty achieving its goal of 66.9 percent mail response rate in 2000.” The

Bureau stated that the 66.9 percent response rate is derived from extensive

testing of its multiple mailing strategy, which includes such innovations as

use of first class postage and a reminder card.

Achieving a 66.9 percent response rate is the Bureau’s best-case scenario

for the 2000 Census. It depends on many traditional census-taking

methods and on new initiatives developed since the 1990 Census,

including the multiple mail strategy, an outreach program that includes

$100 million in paid advertising, and simplified questionnaire forms. The

Bureau’s statement that the 66.9 percent response rate depends primarily

on the multiple mailing strategy is inconsistent with its past statements

and its approach to building high levels of public cooperation with the

census. Thus, we have retained our language that achieving the

66.9 percent mail response goal is dependent upon a successful mailing

Page 45 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 6 

Conclusions

strategy and an effective promotion and outreach program. All the

census-taking methods must work successfully in concert if the Bureau is

to achieve its desired response rate.

With regard to the Bureau’s ability to staff the 2000 Census, the Bureau

agreed with our assessment concerning the magnitude of the challenge

facing the Bureau. However, it disagreed with our characterization that

“the Bureau’s recruitment strategy was based on informal and limited

data.” Given the nature of today’s society, the Bureau’s decision to focus

on evening and weekend hours for conducting nonresponse follow-up

interviews appears reasonable. And, as a result of information

subsequently provided by the Bureau that shows the additional research

on which the Bureau based its recruitment strategy, we agree that the data

were not “informal” and have revised the report accordingly. However, our

conclusion on the limitations of the data were based on Westat’s own

acknowledgment. For example, in one presentation to the Bureau, Westat

suggested that “about 847 enumerators will be needed per district office

out of a pool of 20,300 workers available in any given month.” Westat

stated that this was a “back-of-an-envelope” estimate. In discussing the

pool of workers that may be available to be enumerators, Westat stated:

“At this time, our best evidence for the view that the success of the

[nonresponse follow-up] hinges on tapping into the pool of ‘moonlighters’

comes from Westat’s own hiring practices and those of similar firms.”

The Bureau agreed its sampling and statistical estimation plans pose

certain challenges and added that a nonsampling census plan would

present many of the same challenges as well as many others. The Bureau

agreed with our concerns about the tight scheduling of its nonresponse

follow-up and ICM operations as well as about quality control issues. The

Bureau observed that it will implement a number of initiatives, including

its multiple mail strategy, that are designed to improve census data quality.

The Bureau also noted that evaluations of the 1990 census, including ours,

found that data quality diminishes the farther the collection effort gets

from Census Day. Finally, the Bureau said that ICM is the only proven

method for reducing both undercounts and overcounts, and the only

option for conducting the census within current scheduling constraints.

The Bureau stated that it takes seriously the challenges with regard to

quality control, pointing out that it is developing extensive risk mitigation

strategies for key areas of the census. It stated that it is improving the

quality of software development and using peer review and checklists to

ensure the proper implementation of sampling and estimation

Page 46 GAO/GGD-98-74 Preparations for Dress Rehearsal
Chapter 6 

Conclusions

methodology. The Bureau will be sharing the results of its efforts in the

near future. It remains to be seen whether these efforts reduce the risk to

the census.

In summary, the Bureau recognized that it faces substantial challenges in

conducting an effective census in 2000, but remains confident that it will

be successful. We remain concerned, however, that important questions

remain unanswered at a disturbingly late stage in the census cycle when

the emphasis should be on fine tuning rather than basic revisions to

operational plans.

Page 47 GAO/GGD-98-74 Preparations for Dress Rehearsal
Appendix I 

Comments From the Bureau of the Census

Page 48 GAO/GGD-98-74 Preparations for Dress Rehearsal
Appendix I 

Comments From the Bureau of the Census

Page 49 GAO/GGD-98-74 Preparations for Dress Rehearsal
Appendix I 

Comments From the Bureau of the Census

Page 50 GAO/GGD-98-74 Preparations for Dress Rehearsal
Appendix II 

Major Contributors to This Report

General Government

Division, Washington,

D.C.

James H. Burow, Assistant Director, Federal Management and Workforce

Issues

Robert Goldenkoff, Assistant Director

Don Allison, Senior Evaluator

Victoria Miller, Senior Evaluator

Bruce Taylor, Senior Statistician

Dallas Office Marcia McWreath, Senior Evaluator

Reid Jones, Senior Evaluator

James Rose, Evaluator

(410220) Page 51 GAO/GGD-98-74 Preparations for Dress Rehearsal
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100