a

GAO

United States General Accounting Office

Report to Congressional Requesters

January 2003 DECENNIAL CENSUS

Methods for Collecting 

and Reporting Data on 

the Homeless and 

Others without 

Conventional Housing 

Need Refinement

GAO-03-227
The Bureau's original plan for releasing Service-Based Enumeration data 

was outlined in an April 1999 internal memorandum that called for the 

separate release of data on people counted at "emergency and transitional 

shelters." The Bureau planned to combine other components of ServiceBased Enumeration, including people counted at soup kitchens, regularly 

scheduled mobile food vans, and certain outdoor locations, into a single 

category. Driving the Bureau's decision was its experience during the 1990 

Census when it released separate counts of people found at shelters, on the 

street, and similar locations that proved to be incomplete. The Bureau also 

tried to ensure that the Service-Based Enumeration figures could not be used 

as a "homeless" count, because it was not designed to provide a specific 

count of the homeless. Instead, the operation was part of a larger effort to 

count people without conventional housing. 

The Homeless Are Hard to Enumerate 

Source: U.S. Bureau of the Census. 

In January 2001, the Bureau changed its earlier decision because a statistical 

procedure used to refine the emergency and transitional shelter data proved 

to be unreliable, which lowered the quality of the data. In response, the 

Bureau combined the shelter data with a category called "other noninstitutional group quarters," a category that also includes data on people 

enumerated in several other group locations such as facilities for victims of 

natural disasters. In the fall of 2001, the Bureau produced a heavily qualified 

special report on the shelter data. A key cause of the Bureau's shifting 

position on reporting these data appears to be its lack of well documented, 

transparent, clearly defined, and consistently applied guidelines on the 

minimum quality necessary for releasing data. Had these guidelines been in 

place at the time of the census, the Bureau could have been better 

positioned to make an objective decision on releasing these figures. 

Additionally, the Bureau could have used the guidance to explain to data 

users the reasons for the decision, eliminating any appearance of censorship 

and arbitrariness. Because the Bureau did not always adequately 

communicate its plans for releasing the data, expectation gaps developed 

between the Bureau and entities that helped with Service-Based 

Enumeration. 

DECENNIAL CENSUS 

Methods for Collecting and Reporting 

Data on the Homeless and Others 

Without Conventional Housing Need 

Refinement 

www.gao.gov/cgi-bin/getrpt?GAO-03-227.

To view the full report, including the scope 

and methodology, click on the link above. 

For more information, contact Particia A. 

Dalton at (202) 512-6806. 

Highlights of GAO-03-227, a report to 

Congressional Requesters 

January 2003 

The Bureau of the Census 

partnered with local governments, 

advocacy groups, and other 

organizations to help it enumerate 

people without conventional 

housing. Counting this 

population—which includes shelter 

residents and the homeless—has 

been a longstanding challenge for 

the Bureau. A number of 

organizations put substantial 

resources into an operation the 

Bureau called Service-Based 

Enumeration. In return, some 

expected the Bureau to provide 

data that would help them plan and 

deliver employment, health, and 

other services. However, the 

Bureau did not release the data as 

planned, which raised questions 

about the Bureau’s decisionmaking on data quality issues. In 

response to a congressional 

request, GAO examined the 

Bureau’s decision-making process 

behind its change in plans. 

The Secretary of Commerce should 

direct the Bureau to (1) properly 

test and evaluate procedures for 

counting people without 

conventional housing; (2) develop 

guidelines for decisions on the 

level of quality needed to release 

data to the public, how to 

characterize any limitations, and 

when it is acceptable to suppress 

data; and (3) ensure that plans for 

releasing data are clearly 

communicated to data users. 

The Bureau agreed with GAO’s 

recommendations, but took issue 

with our findings on the adequacy 

of its data quality guidelines. 
Page i GAO-03-227 Decennial Census

Contents

Letter 1

Results In Brief 2

Background 4

Scope and Methodology 5

The Bureau of the Census Twice Changed Plans for Reporting 

Service-Based Enumeration Data 6

Census Bureau Had Few Documented Guidelines Governing the 

Release of Census Data 14

Conclusions 15

Recommendations for Executive Action 16

Agency Comments and Our Evaluation 17

Appendix

Appendix I: Comments from the Secretary of Commerce 20

Related GAO Products 

on the Results of the 

2000 Census and 

Lessons Learned for a 

More Effective Census 

in 2010

30

Figure Figure 1: The Bureau Changed Its Original Plan to Release 

Emergency and Transitional Shelter Data and Combined 

Them with Other Noninstitutional Group Quarters Data 8

This is a work of the U.S. Government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. It may contain copyrighted graphics, images or other materials. 

Permission from the copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAO’s product.
Page 1 GAO-03-227 Decennial Census

United States General Accounting Office

Washington, D.C. 20548

A

January 17, 2003 Letr

The Honorable Henry A. Waxman

Ranking Minority Member

Committee on Government Reform

House of Representatives

The Honorable Danny K. Davis

Ranking Minority Member

Subcommittee on Civil Service, 

Census and Agency Organization 

Committee on Government Reform

House of Representatives

The Honorable Carolyn B. Maloney

The Honorable Dennis J. Kucinich

The Honorable Wm. Lacy Clay

House of Representatives

For the 2000 Census, the Bureau of the Census employed several initiatives 

to help ensure a complete and accurate count of people without 

conventional housing. Enumerating this segment of the population, which 

contains, among others, people referred to as “homeless,” has been an 

ongoing problem for the Bureau. In one initiative, known as Service-Based 

Enumeration, census enumerators attempted to count these individuals at 

emergency and transitional shelters, soup kitchens, regularly scheduled 

mobile food vans, as well as at what the Bureau calls “targeted nonsheltered outdoor locations” (TNSOL) for people living on the street at 

targeted locations who do not use services. To help locate and count 

people, the Bureau partnered with organizations providing services to the 

homeless and local governments, some of which put substantial resources 

into their efforts. In return, some of these organizations expected the 

Bureau to provide data that would help them plan and deliver health, 

employment, and other services directed toward this population. 

However, in its review of the emergency and transitional shelter data, the 

Bureau identified serious concerns with the quality of the data and 

concluded that the data should not be released without explanation of their 

extensive limitations and caveats. As a result, the Bureau decided not to 

separately report the emergency and transitional shelter data in the initial 

release of summary files as originally planned. Instead, the Bureau 

combined the emergency and transitional shelter data with a category 
Page 2 GAO-03-227 Decennial Census

called “other non-institutional group quarters.” At the census tract level 

(small statistical subdivisions of counties) this category included soup 

kitchens, regularly scheduled mobile food vans, shelters for victims of 

domestic violence, residential care facilities providing protective oversight, 

staff quarters including those for nurses and interns at military and general 

hospitals, and living quarters for victims of natural disasters. At the block 

level, in addition to the categories listed above, “other non-institutional 

group quarters” included group homes, religious group quarters, other 

nonhousehold living situations, and workers’ dormitories. Aggregating the 

shelter numbers with these other data raised concerns among some data 

users that the Bureau was suppressing the results.

You asked us to examine the Bureau’s decision-making process behind its 

change in plans. As agreed with your offices, this report examines (1) the 

Bureau’s plans for reporting the results of Service-Based Enumeration and 

its reasons for changing those plans and (2) the Bureau’s protocols for 

releasing data. 

Members of the Congress also raised concerns about the quality of 

Hispanic subgroup data and asked us to review the Bureau’s decisionmaking process for collecting and reporting ethnicity information. The 

results of that study are included in a companion report.1

 Both reports are 

part of our ongoing series on the lessons learned from the 2000 Census that 

can help inform the planning effort for 2010. (See the Related GAO 

Products section for a list of reports issued to date on census issues.)

Results in Brief The Bureau’s original plan for disseminating Service-Based Enumeration 

data was outlined in an April 1999 internal memorandum that called for the 

separate release of data on “emergency and transitional shelters,” but did 

not specify why the Bureau was not separately releasing data on the other 

locations enumerated during the Service-Based Enumeration—soup 

kitchens, regularly scheduled mobile food vans, shelters for victims of 

domestic violence, and targeted nonsheltered outdoor locations. The 

Bureau’s plan reflected its experience during the 1990 Census when it 

released separate counts of people found at emergency shelters, street 

locations, and similar locations. However, those counts proved to be 

incomplete. The Bureau indicated from the beginning that these 1990 

1

 U.S. General Accounting Office, Decennial Census: Methods for Collecting and Reporting 

Hispanic Subgroup Data Need Refinement, GAO-03-228 (Washington, D.C.: Jan. 17, 2003).
Page 3 GAO-03-227 Decennial Census

counts could not be added together to produce a count of the homeless 

population. Despite the Bureau’s warnings to the contrary, the data were 

often misinterpreted as a “homeless” count. Thus, in developing its data 

release plans, the Bureau took steps to ensure that the Service-Based 

Enumeration figures could not be added together and used as a homeless 

count.

In January 2001, the Bureau changed its earlier decision to include the data 

on emergency and transitional shelters in one of its early data releases 

because a procedure used to refine the Service-Based Enumeration data 

proved to be unreliable. Although the Bureau had tested the procedure 

earlier in the decade, because of methodological limitations the test did not 

reveal any flaws. However, because problems with the procedure surfaced 

during the review of 2000 Census operations, the Bureau decided to 

combine the emergency and transitional shelter data with the “other noninstitutional group quarters” category that also includes data on people 

enumerated in the other categories of the Service-Based Enumeration and 

in several other group locations, such as facilities for victims of natural 

disasters. As a result of this decision, the Bureau did not separately report 

any data from the Service-Based Enumeration in its initial release of 

Census 2000 data. These were the only data with separate reporting 

categories that the Bureau decided to collapse into another category.

In the fall of 2001, the Bureau produced a special report on the emergency 

and transitional shelter data—including most of the same data that the 

Bureau earlier stated it could not release because of quality concerns. This 

report did not include data on targeted nonsheltered outdoor locations, or 

on soup kitchens and mobile food vans. The Bureau added a lengthy 

discussion of the limitations of the data and emphasized that they should 

not be interpreted as a count of the homeless population. 

Although the Bureau worked closely with a number of government entities, 

advocacy groups, and other organizations to conduct Service-Based 

Enumeration, reconciling its often competing data needs proved 

challenging. Compounding the Bureau’s difficulties, expectation gaps 

developed between these entities and the Bureau because the Bureau did 

not always clearly and consistently communicate its plans. 

A key cause of the Bureau’s shifting position on reporting the ServiceBased Enumeration data appears to be its lack of clear, documented, and 

consistently applied guidelines governing the release of data from the 2000 

Census. Had these guidelines been in place at the time of the census, they 
Page 4 GAO-03-227 Decennial Census

could have helped Bureau managers decide whether to release the ServiceBased Enumeration data and how to characterize these data. Additionally, 

the Bureau could use the guidelines to defend its decisions once they were 

made, thus helping to ensure that the Bureau’s decisions both are, and 

appear to be, completely objective. 

To ensure that the 2010 Census will provide data users with more complete, 

accurate, and useful information on people without conventional housing, 

we recommend that the Secretary of Commerce direct the Bureau of the 

Census to ensure that the procedures for enumerating and estimating 

segments of the population without conventional housing are properly 

tested and evaluated under conditions as similar to the census as possible. 

In addition, the Bureau should develop clearly documented, transparent, 

and consistently applied agencywide guidelines for releasing all census 

data to the public and ensure that plans for releasing data are clearly and 

consistently communicated to the public.

The Secretary of Commerce forwarded written comments from the Bureau 

of the Census on a draft of this report (see app. I). The Bureau agreed with 

our recommendations and is taking steps to implement them, but took 

exception to our findings concerning the adequacy of its data quality 

guidelines and communication with the public.

Background The procedures the Bureau used during the 1990 Census to count people 

without conventional housing had limitations that resulted in incomplete 

data.2

 To address these limitations and help improve the quality of the data, 

the Bureau used a procedure for the 2000 Census called Service-Based 

Enumeration that attempted to count people where they receive services 

such as emergency shelters, soup kitchens, and regularly scheduled mobile 

food vans. Service-Based Enumeration also counted people in targeted 

nonsheltered outdoor locations such as encampments beneath bridges. 

The operation occurred from March 27 through March 29, 2000.3

 

2

 For further information see, U.S. General Accounting Office, 1990 Census: Limitations in 

Methods and Procedures to Include the Homeless, GAO/GGD-92-1 (Washington, D.C.: 

Dec. 30, 1991).

3

 For information on the conduct of Service-Based Enumeration see, U.S. General 

Accounting Office, 2000 Census: Progress Report on the Mail Response Rate and Key 

Operations, GAO/T-GGD/AIMD-00-136 (Washington, D.C.: Apr. 5, 2000).
Page 5 GAO-03-227 Decennial Census

According to Bureau officials, Service-Based Enumeration was not 

designed, and was never intended, to provide a specific count of homeless 

persons. Instead, the operation was part of a larger effort to count people 

without conventional housing, including people in “institutional group 

quarters” such as correctional facilities, nursing homes, and mental 

hospitals, and “non-institutional group quarters” such as college 

dormitories, military quarters, and group homes. Service-Based 

Enumeration counted people in specific categories of noninstitutional 

group quarters. 

To help ensure a complete count of people without conventional housing, 

the Bureau partnered with local governments and community advocacy 

groups to obtain lists of service locations and to assist with the 

enumeration.4 In some cases, the Bureau hired clients of the advocacy 

groups and other people trusted by the homeless to conduct Service-Based 

Enumeration. For example in Atlanta, an advocacy group for homeless 

veterans helped the Bureau employ homeless veterans to improve the 

count of this population. Local governments helped the Bureau as well, 

often investing considerable resources. For example, Los Angeles paid to 

keep its city-run shelters open on the night they were enumerated so that 

people using their services could be counted.

Scope and 

Methodology

To address your concerns about the Bureau’s dissemination of data on 

persons without conventional housing, we agreed to examine (1) the 

Bureau’s plans for reporting the results of Service-Based Enumeration and 

its reasons for changing those plans and (2) the Bureau’s protocols for 

releasing data. To accomplish these objectives, we interviewed key Bureau 

officials and reviewed relevant Bureau documents and data such as 

operational plans, decision memorandums, and the Bureau’s partnership 

program evaluation. 

In order to obtain the perspective of data users, partners, and stakeholders, 

we conducted in-person and telephone interviews with homeless 

advocates, local government officials, and representatives of public service 

agencies in New York City, Los Angeles, Cleveland, Atlanta, and 

Washington, D.C. These cities had large numbers of people without 

4

 For additional information see, U.S. General Accounting Office, 2000 Census: Review of 

Partnership Program Highlights Best Practices for Future Operations, GAO-01-579

(Washington, D.C.: Aug. 20, 2001).
Page 6 GAO-03-227 Decennial Census

conventional housing and they were actively involved with the Bureau 

during the 2000 Census. The organizations we contacted also provided 

relevant documentation, such as comprehensive file documents relating to 

partnership activities. 

In addition to the above locations, we did our audit work at Bureau 

headquarters in Suitland, Maryland. Our audit work was conducted from 

April 2002 through September 2002 in accordance with generally accepted 

government auditing standards. 

We requested comments on a draft of this report from the Secretary of 

Commerce. On November 21, 2002, the Secretary forwarded the Bureau’s 

written comments on the draft (see app. I). We address these comments at 

the end of this report.

The Bureau of the 

Census Twice Changed 

Plans for Reporting 

Service-Based 

Enumeration Data

Under the Bureau’s original plan for releasing Service-Based Enumeration 

data in Summary File-1 (SF-1), 5

 the emergency and transitional shelter 

count was one of several categories of noninstitutional group quarters data 

that were to be reported separately. Other people counted in the ServiceBased Enumeration, including people counted at targeted nonsheltered 

outdoor locations, soup kitchens, and regularly scheduled mobile food 

vans, were to be combined and reported under the category “other noninstitutional group quarters.” This category also included residential care 

facilities providing protective oversight, shelters against domestic violence, 

staff dormitories for nurses and interns at military and general hospitals, 

and living quarters for victims of natural disasters. 

5

 The SF-1 is the summary file in which the Bureau presents population and housing data for 

the total population. Other than the Census 2000 Redistricting Data Summary File, the SF-1 

file is the first product released after the census. 
Page 7 GAO-03-227 Decennial Census

This decision was documented in an April 1999 internal memorandum from 

the Bureau’s Assistant Division Chief for Special Population Statistics to 

the Assistant Division Chief for Census Programs. The Service-Based 

Enumeration operation took place a year later, in March 2000. The April 

1999 plan was in large part a reaction to the challenges the Bureau faced 

counting the emergency shelter and street population during the 1990 

Census. Although the Bureau disseminated separate counts of people 

found at emergency shelters, preidentified street locations, and similar 

sites, the counts proved to be incomplete. 6

 Moreover, the Bureau stated in 

its October 2001 report that despite its warnings to the contrary, the data 

were sometimes misinterpreted as a “homeless” count. 

The October report does not offer an example of this, but the 

misinterpretation clearly played a role in a lawsuit against the Bureau. 7

 As 

a result, when designing the 2000 census, the Bureau attempted to both 

improve the count and take precautions to ensure that the Service-Based 

Enumeration count would not be misconstrued as a count of the homeless.

The Bureau’s data dissemination plans took into account the 

recommendations of the Commerce Secretary’s 2000 Census Advisory 

Committee, a panel that included representatives of advocacy and other 

groups (including representatives from organizations that represent local 

governments) that met periodically to review the Bureau’s plans. The 

homeless population was represented by the National Coalition for the 

Homeless—an advocacy group that coordinates a network of 300 state and 

local housing and homeless organizations. In its January 1999 final report, 

the Census 2000 Advisory Committee recommended that special attention 

be paid to tabulating the results of Service-Based Enumeration and 

targeted outdoor enumerations so that they could not be aggregated and 

used as a homeless count. 

6

 For more information on the 1990 count see, U.S. General Accounting Office, Counting the 

Homeless: Limitations of 1990 Census Results and Methodology, GAO/T-GGD-91-29

(Washington, D.C.: May 9, 1991) and 1990 Census: Limitations in Methods and Procedures 

to Include the Homeless, GAO/GGD-92-1 (Washington, D.C.: Dec. 30, 1991).

7

 One example was the National Law Center On Homelessness and Poverty, et. al., v. Brown, 

where plaintiffs alleged that the Department of Commerce effectively excluded the nation’s 

homeless population from the 1990 decennial census in violation of the Constitution and the 

Administrative Procedure Act. The District Court denied the plaintiff’s claim (1994 WL 

521334) (D.D.C. Sept. 15, 1994) and the Court of Appeals dismissed the lawsuit for lack of 

standing. 91 F.3d 178 (D.C. Cir. 1996). 
Page 8 GAO-03-227 Decennial Census

The Bureau Changed Its 

Dissemination Plans 

Because of Data Quality 

Concerns 

In January 2001, 5 months before the SF-1 release, the Bureau reversed its 

April 1999 decision to release emergency and transitional shelter data 

separately because of “data quality concerns.” Instead, as shown in figure 

1, the Bureau planned to combine the emergency and transitional

Figure 1: The Bureau Changed Its Original Plan to Release Emergency and Transitional Shelter Data and Combined Them with 

Other Noninstitutional Group Quarters Data

■ Correctional institutions

■ Nursing homes

■ Hospitals

■ Juvenile institutions

Institutionalized population 

reporting categories

Group quarters

Non-Institutionalized Population

■ College dorms

■ Military quarters

■ Group homes

■ Religious group quarters

■ Worker dormitories

■ Crews of maritime vessels

■ Emergency and transitional shelters

■ Other noninstitutional group quarters

The other noninstitutional group quarters categories are:

Soup kitchens and regularly scheduled food vans

Targeted nonsheltered outdoor locations

Residential facilities "providing protective oversight"

Living quarters for victims of natural disasters

Staff residents of institutions

 Shelters for victims of domestic violence

Revised plan (January 2001) 

Noninstitutionalized population 

reporting categories

The other noninstitutional group quarters categories are:

Emergency and transitional shelters

Soup kitchens and regularly scheduled food vans

Targeted nonsheltered outdoor locations

Residential facilities "providing protective oversight"

Living quarters for victims of natural disasters

Staff residents of institutions

 Shelters for victims of domestic violence

Group quarters

■ College dorms

■ Military quarters

■ Group homes

■ Religious group quarters

■ Worker dormitories

■ Crews of maritime vessels

■ Other noninstitutional group quarters

Noninstitutionalized population 

reporting categories

Original plan (April 1999)

Source: GAO depiction based on Bureau of the Census data.
Page 9 GAO-03-227 Decennial Census

shelter data with the “other non-institutional group quarters.” This 

category contained data on a variety of living arrangements including 

facilities for natural disaster victims. The Bureau’s decision was contained 

in an internal Bureau memorandum from the Chief of the Population 

Division to the Chief of the Decennial Systems and Contracts Management 

Office. Bureau officials told us that the decision to exclude a separate 

emergency and transitional shelter count in SF-1 was made between 

December 2000 and January 2001, by the Director of the Decennial Census 

with input from the Associate Director Decennial Census, the Population 

Division, the Associate Director for Demographic Programs, the Decennial 

Management Division, and the Decennial Statistical Studies Division.8

 

According to Bureau officials, their concerns focused on the accuracy of a 

new statistical procedure called “multiplicity estimation” that adjusted the 

number counted to better reflect the number of actual shelter users. 

Because Service-Based Enumeration only counted people who were at 

these facilities on the day of enumeration, the Bureau intended to use 

multiplicity estimation to calculate the number of people who used these 

facilities but were not present during Service-Based Enumeration. The 

multiplicity estimation procedure was based on information from those 

who were counted and on the number of times they used the service 

facilities in the prior week. An estimate of people not counted on the day 

of enumeration was added to the count of people who were. According to 

the Bureau, the multiplicity estimates tested well during the 1998 dress 

rehearsal for the 2000 Census possibly because the three rehearsal sites did 

not offer large enough sample sizes of the appropriate populations to 

adequately test this procedure.9

 However, during the 2000 Census the 

Bureau found that a census question pertaining to facility usage upon 

which the multiplicity estimates were based had a low response rate. 

Moreover, the Bureau found that respondents, particularly in shelters, did 

not answer the question accurately. Due to data quality concerns, the 

Bureau decided not to use multiplicity estimation to adjust the data and 

consequently decided not to report the data separately.

8

 Although the Bureau changed its plan in January 2001, the technical documentation for

SF-1, released at the same time, still indicated that emergency and transitional shelter data 

would be separately reported.

9

 The dress rehearsal for the 2000 Census was conducted in Sacramento, California, City of 

Columbia, South Carolina, and Menominee County, Wisconsin, including the Menominee 

Indian Reservation. The dress rehearsal was designed to test the overall design of the 2000 

Census.
Page 10 GAO-03-227 Decennial Census

Bureau officials said they did not announce the change in plans because 

they were still evaluating the problems with the data. It was not until June 

2000 that the Bureau began recalculating the data and making a final 

decision on which categories to aggregate. Ultimately, the Bureau did not 

report any of the Service-Based Enumeration data separately in SF-1. 

Emergency and transitional shelter data were the only data that were to be 

released in SF-1 under separate reporting categories that the Bureau 

decided to combine with another category.

The Bureau Produced a 

Special Report in October 

2001 on the Emergency and 

Transitional Shelter 

Population

The release of the SF-1 data in June 2001 produced public discussion in the 

press, among census partners, and in the Congress about the Bureau’s 

decision to not separately release Service-Based Enumeration data. In a 

briefing for staff of the House Committee on Governmental Affairs, the 

Associate Director of the Decennial Census announced that the Bureau 

planned to produce a separate report on the emergency and transitional 

shelter data. In October 2001, the Bureau issued a special report, entitled 

Emergency and Transitional Shelter Population: 2000. This report 

separately identified emergency and transitional shelter data for various 

levels of geography down to the census tract level with 100 or more people 

in emergency and transitional shelters. The report did not include data for 

the populations in targeted nonsheltered outdoor locations, soup kitchens, 

regularly scheduled mobile food vans, and shelters for domestic violence. 

The 17-page report contains an extensive discussion on the limitations of 

the data. For example, the Bureau noted that the data in the report should 

not be construed as a count of people without conventional housing. 

Moreover, the emergency and transitional shelter data at the census tract 

level are not in the hard copy, but rather in the Internet version of the 

report.10 The Bureau stated that all Census 2000 data at the tract level are 

available on the Internet and are not available in printed reports.

The October report contains most of the same data that were to be released 

under the April 1999 dissemination plan for SF-1. The Bureau asserted that 

the data quality concerns with the emergency and transitional shelter data 

(cited when it changed the plan to release these data in SF-1) required that 

the data be presented in a manner that allowed the Bureau to clearly 

10 U.S. Bureau of the Census, Population in Emergency and Transitional Shelters 

(PHC-T-12) (Washington, D.C.: Oct. 30, 2001). 

http://www.census.gov/population/www/cen2000/phc-t12.html
Page 11 GAO-03-227 Decennial Census

outline the data’s limitations. The October 2001 report contained an 

extended discussion of these limitations.

The October 2001 report also identified reasons the Bureau did not (and 

never planned to) separately release data on people counted at targeted 

nonsheltered outdoor locations, soup kitchens, regularly scheduled mobile 

food vans, and shelters for victims of domestic violence, including the 

following.

• People without conventional housing who were at outside locations 

other than the targeted nonsheltered outdoor locations identified for the 

census were not included in the TNSOL operation. 

• For the purposes of the TNSOL operation, the definition of “outdoor” 

excluded both mobile and transient locations used by people 

experiencing homelessness as well as abandoned buildings. 

• The option was given to the individuals found at soup kitchens and 

regularly scheduled mobile food vans to select “usual home elsewhere.” 

For example, if an individual enumerated at a soup kitchen listed a usual 

home elsewhere, then that person was tabulated at their usual residence 

and not at the service location. Therefore, the data on this population 

would not reflect a true count of the individuals using these services. 

Prior to publicly releasing the October report, the Bureau asked two 

representatives from the National Coalition for the Homeless to review a 

draft of the portion of the report that described the limitations of the data. 

The National Coalition for the Homeless commented extensively on the 

section containing the caveats and limitations in order to strengthen the 

report. A member of the Board of Directors for the National Coalition for 

the Homeless told us that he provided this feedback both as an 

academician and a stakeholder. Bureau officials stated that because of its 

position on the Bureau’s Census Advisory Committee, the National 

Coalition for the Homeless was the only advocacy group that reviewed any 

portion of the October 2001 report prior to its publication. 

Meeting Data Users’ Needs 

Proved Challenging 

The controversy surrounding the release of the combined Service-Based 

Enumeration data highlights the challenges the Bureau faced in 2000 trying 

to meet the needs of various data users and the work the Bureau still needs 

to do when planning for the 2010 Census to better reconcile those needs. 

For example, several organizations we contacted favored the separate 
Page 12 GAO-03-227 Decennial Census

release of the Service-Based Enumeration data categories. Indeed, local 

government officials we talked to in New York City believed that the data 

would help with grant applications, projections about future service needs, 

and determining their success in getting people off the streets and into 

shelters. The Executive Director of the Northeast Ohio Coalition for the 

Homeless stated that the city of Cleveland does not do its own count of this 

population and, therefore, the Bureau numbers are the only ones available 

on this segment of the population. Los Angeles city officials wanted the 

Service-Based Enumeration data so they could better target their services 

and, like Cleveland, Los Angeles did not have its own data. Several of these 

entities stated that the potential misuse of data was not a valid reason for 

not separately releasing data.

In addition, the majority of the organizations we contacted partnered with 

the Bureau anticipating that they would be able to use the Service-Based 

Enumeration data to evaluate whether improvements were made in 

enumerating local populations without conventional housing in 2000 

compared to 1990. The Assistant City Attorney of Los Angeles estimated 

that Los Angeles spent about $300,000 on the effort to improve the count of 

Los Angeles’s people without conventional housing. For example, as part 

of an extensive effort to help the Bureau develop a list of targeted 

nonsheltered outdoor locations, the city provided senior Bureau staff with 

a helicopter tour over some outdoor locations where people without 

conventional housing lived. The Assistant City Attorney of Los Angeles 

stated that she believed the city would get the targeted nonsheltered 

outdoor locations data that they helped collect and wanted to review. In 

addition, because of the Bureau’s focus on counting people at shelters, the 

city kept shelters open on the night of the enumeration at its own expense 

even though shelters in Los Angeles typically do not have many people 

during warm weather. Los Angeles expected to have detailed data to use to 

evaluate the effectiveness of its resource allocation. 

However, the National Coalition for the Homeless and other advocates of 

the homeless opposed the separate release of any of the Service-Based 

Enumeration data. They were concerned that these data could be misused 

as a count of the homeless population and lead to flawed decision-making 

by policymakers. 

Ultimately, the Bureau left a number of data users unsatisfied. Those who 

wanted the Service-Based Enumeration categories released separately did 

not feel the Bureau met their expectations with the data released in SF-1 or 

with the release of the October report. Users who opposed the separate 
Page 13 GAO-03-227 Decennial Census

release of the data and were pleased that SF-1 combined the Service-Based 

Enumeration components with other data were displeased that the October 

2001 report was released. 

The difficulties the Bureau experienced trying to reconcile the competing 

needs and interests of data users illustrates the importance of effective 

communication between the Bureau and its key data users and partners to 

ensure no expectation gaps develop. More than just a good business 

practice, federal internal control standards require agencies to have 

effective external communications with groups that can have a serious 

impact on programs, projects, operations, and other activities.11 

However, our conversations with several Bureau partners and our review 

of Bureau documents suggest that communications were sometimes vague 

and insufficient. For example, although the April 1999 memorandum that 

outlined the Bureau’s initial data dissemination plans was written a year 

before the 2000 Census, this information may not have been effectively 

communicated to the Bureau’s partners. Indeed, at a Capitol Hill briefing 

on this topic in June 2001, Bureau officials themselves acknowledged that 

they did not do a good job of communicating on this issue. Some of the 

partners we spoke to indicated that had they known earlier about the 

Bureau’s plans to limit the release of Service-Based Enumeration data they 

might have focused their resources on different census operations. 

Further, our review of Bureau documents indicated that the information on 

the “official plan” for the release of the different Service-Based 

Enumeration categories of data was limited and inconsistent. Some 

partners stated that they did not know that the Bureau never intended to 

report the targeted nonsheltered outdoor location data. 

Although the Bureau made numerous presentations on Service-Based 

Enumeration that emphasized there would be no count of the homeless, 

the Bureau provided little detail on how components of Service-Based 

Enumeration would actually be presented. In the absence of clear 

communication from the Bureau, partners developed their own 

expectations of what would be released. Several of the local officials and 

advocates that we spoke to expected that the data would be released in the 

11 U.S. General Accounting Office, Internal Control Standards, Internal Control 

Management and Evaluation Tool (August 2001) and U.S. General Accounting Office, 

Internal Control Standards for Internal Control in the Federal Government (November 

1999).
Page 14 GAO-03-227 Decennial Census

same detail as it was in 1990, because they were not told otherwise. For 

example, a Los Angeles government official said that the Bureau stated it 

would not provide a homeless count in 1990, but it still released the street 

count data separately. By focusing resources on counting specific 

categories of the population, the Bureau may have created expectations 

that there would be a count of that population. 

Census Bureau Had 

Few Documented 

Guidelines Governing 

the Release of Census 

Data 

A cause of the Bureau’s shifting position on reporting the components of 

Service-Based Enumeration appears to be its lack of documented, clear, 

transparent, and consistently applied guidelines governing the release of 

data from the 2000 Census. Except for some guidance aimed at protecting 

the confidentiality of census records, the Bureau had few written 

guidelines on the level of quality needed to release data to the public. Had 

these guidelines been in place during the decennial census, they could have 

informed the Bureau’s decision on whether to release the Service-Based 

Enumeration data, how to characterize these data, and help defend the 

decision after it was made. Such guidelines could also provide a basis 

ahead of time for expectations about the conditions under which data will 

or will not be released.

Although Bureau officials emphasized that the Bureau has a long tradition 

of high standards and procedures that yield quality data (to its credit, the 

Bureau’s quality assurance practices identified the problem with the 

multiplicity estimator), the officials acknowledged that these standards 

were primarily part of the agency’s institutional knowledge. The written 

guidance that did exist appeared to be vague and insufficient for making 

consistent decisions on the quality thresholds needed for releasing data to 

the public, and the circumstances under which it might be appropriate to 

suppress certain data. 
Page 15 GAO-03-227 Decennial Census

According to the Bureau’s Associate Director for Methodology and 

Standards, a technical paper issued in 1974 and revised in 1987 contained 

the Bureau’s only written guidelines for discussion and presentation of 

errors in data. This paper noted that, “[e]stimates for individual cells of a 

published table should not be suppressed solely because they are subject to 

large sampling errors or large nonsampling variances, provided users are 

given adequate caution of the lack of reliability of the data. On the other 

hand, data known to have very serious bias may be suppressed.”12

A Newly Created Bureau 

Program Could Provide 

Guidelines for Releasing 

Data

In 2000, the Bureau initiated a new quality assurance program to document 

Bureau-wide protocols designed to ensure the quality of data collected and 

disseminated by the Bureau. The Bureau’s Methodology and Standards 

Council is charged with setting statistical and survey quality standards and 

guidelines for Bureau surveys and censuses. In support of this role, the 

council has established a quality framework in which the demographic, 

economic, and decennial areas can share and support common principles, 

standards, and guidelines. The quality framework covers eight unique 

areas, one of which is dissemination.

Because this Bureau program is in its initial stages, we could not evaluate 

it. However, Bureau officials believe that the program is a significant first 

step in addressing the lack of agencywide written guidelines for releasing 

data. The initiative appears to be consistent with Office of Management 

and Budget guidelines issued in February 2002 requiring federal agencies to 

issue their own guidance for ensuring and maximizing the quality, 

objectivity, utility, and integrity of information disseminated by the 

agency.13 As the Bureau develops its guidelines, it will be important that 

they be well documented, transparent, clearly defined, and consistently 

applied.

Conclusions Although Service-Based Enumeration was designed to address the 

challenges the Bureau encountered during the 1990 Census in obtaining a 

complete count of people without conventional housing, the Bureau’s 

experience during the 2000 Census suggests that tallying this population 

12 U.S. Department of Commerce, Technical Paper 32, Standards for Discussion and 

Presentation of Errors in Data (March 1974), p. 3. 

13 Issued at 67 Fed. Reg. 8452 (Feb. 22, 2002).
Page 16 GAO-03-227 Decennial Census

group remains problematic. Moreover, the Bureau’s difficulties were 

compounded by its shifting position on how to report the data once they 

were collected. A number of government, community, and advocacy 

organizations helped the Bureau enumerate this population group. 

However, the Bureau, by first planning to release the data one way, then 

changing the decision, and ultimately releasing the data anyway—all for 

reasons that were not clearly articulated to the Bureau’s stakeholders—

raised questions about the Bureau’s decision-making on data quality issues. 

As noted at the beginning of this report, related questions have also been 

raised about how the Bureau collected and reported data on Hispanic 

subgroups. To the extent similar incidents occur in the future, they could 

undermine public confidence in the accuracy and credibility of Bureau 

data.

Thus, as the Bureau plans for the 2010 Census, it will be important for it to 

refine its methods for enumerating people living in unconventional housing 

and reporting the resulting data, in part by properly testing and evaluating 

those methods. As noted earlier, the Bureau could not properly test a key 

statistical technique during the census dress rehearsal because the sample 

size was too small.

Moreover, while addressing the competing needs and desires of data users 

will likely remain a considerable challenge, it will be important for the 

Bureau to more effectively articulate its plans to avoid the expectation gaps 

that occurred during 2000. The Bureau’s plans for collecting data on 

persons without conventional housing need to specify how the Bureau 

plans to separately report these data.

Bureau-wide guidelines on the level of quality needed to release data to the 

public, on how and when to document data limitations, and on the 

circumstances under which it is acceptable to suppress data, could help the 

Bureau be more accountable and consistent in its dealings with data users 

and stakeholders, and help ensure that the Bureau’s decisions both are, and 

appear to be, totally objective. 

Recommendations for 

Executive Action

To ensure that the 2010 Census will provide public data users with more 

complete, accurate, and useful information on the segment of the 

population without conventional housing, we recommend that the 

Secretary of Commerce direct the Director of the Bureau of the Census to 

do the following.
Page 17 GAO-03-227 Decennial Census

1. Ensure that all procedures for enumerating and estimating segments of 

the population without conventional housing are properly tested and 

evaluated under conditions as similar to the census as possible.

2. Develop agencywide guidelines for Bureau decisions on the level of 

quality needed to release data to the public, how to characterize any 

limitations in the data, and when it is acceptable to suppress the data 

for reasons other than protecting the confidentiality of respondents. 

Ensure that these guidelines are documented, transparent, clearly 

defined, and consistently applied.

3. Ensure that the Bureau’s plans for releasing data are clearly and 

consistently communicated with the public. 

Agency Comments and 

Our Evaluation

The Secretary of Commerce forwarded written comments from the Bureau 

of the Census on a draft of this report (see app. I). The Bureau agreed with 

each of our recommendations and, as indicated in the letter, is taking steps 

to implement them. However, it expressed several general concerns about 

our findings. The Bureau’s principal concerns and our response are 

presented below. The Bureau also suggested minor wording changes to 

provide additional context and clarification. We accepted the Bureau’s 

suggestions and made changes to the text as appropriate. 

The Bureau took exception to our findings concerning the adequacy of its 

data quality guidelines, noting that the Bureau’s decisions regarding the 

release and characterization of emergency and transitional shelter data 

were based on established guidelines for data quality. However, the Bureau 

did not cite any written guidelines to support its position. As noted in our 

report, Bureau officials, including the Associate Director for Methodology 

and Standards, told us that the Bureau had few written guidelines, 

standards, or procedures related to the quality of data released to the 

public. In this report we acknowledge the Bureau’s tradition of high 

standards and procedures that yield quality data. However, according to 

the Bureau, these standards are generally undocumented and part of the 

agency’s institutional knowledge. To provide a basis for consistent 

decision-making and clear communication within the Bureau and to the 

public, guidelines on the quality of data released to the public must be fully 

documented, transparent, clearly defined, and consistently applied. 

Additionally, the Bureau said that when data do not meet an acceptable 

level of quality, it considers various options for modifying its dissemination 
Page 18 GAO-03-227 Decennial Census

plans. The Bureau’s decision to delay the release of the emergency and 

transitional shelter data may have been entirely appropriate. Our concern 

is not that the Bureau changed its plans, but that it could not provide us its 

guidelines for determining an acceptable level of quality or clearly indicate 

how it determined that the data did not meet minimal quality standards for 

release in SF-1. 

The Bureau also commented that its decisions regarding the distribution of 

data from SF-1 were well publicized and that the only change in Bureau 

plans for the release of Service-Based Enumeration data was the decision 

to delay release of the emergency and transitional shelter data. This report 

focused on the changing plans for the release of the emergency and 

transitional shelter data and noted that the Bureau never intended to 

release any other data from the Service-Based Enumeration. However, we 

found that the Bureau did not effectively communicate its decisions with 

its partners or the public. Decisions on the release of the emergency and 

transitional shelter data were contained in internal decision memoranda. 

We found that these decisions were not always reflected in new releases of 

the SF-1 documentation. Although Bureau officials told us that they always 

intended to produce a separate report on emergency and transitional 

shelter data, they did not make this intention public when the SF-1 data 

were released. Some stakeholders did not realize that the Bureau was not 

releasing emergency and transitional shelter data with SF-1 until they 

examined the SF-1 data. As we stated in our report, these communication 

problems can undermine stakeholder and public confidence in the Bureau 

and its products. 

As agreed with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days from its 

issue date. At that time, we will send copies of this report to the Chairman 

of the House Committee on Government Reform, the Chairman of the 

Subcommittee on Civil Service, Census and Agency Organization, the 

Secretary of Commerce, and the Director of the Bureau of the Census. 

Copies will be made available to others on request. This report will also be 

available at no charge on GAO’s home page at http://www.gao.gov.
Page 19 GAO-03-227 Decennial Census

Please contact me on (202) 512-6806 or by e-mail at daltonp@gao.gov if you 

have any questions. Other key contributors to this report were Robert 

Goldenkoff, Timothy Wexler, Elizabeth Powell, Chris Miller, James 

Whitcomb, Ty Mitchell, Robert Parker, and Michael Volpe.

Patricia A. Dalton

Director 

Strategic Issues
Page 20 GAO-03-227 Decennial Census

Appendix I

Comments from the Secretary of Commerce AAppenednxidexiIs
Appendix I

Comments from the Secretary of Commerce

Page 21 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 22 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 23 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 24 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 25 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 26 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 27 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 28 GAO-03-227 Decennial Census
Appendix I

Comments from the Secretary of Commerce

Page 29 GAO-03-227 Decennial Census
Page 30 GAO-03-227 Decennial Census

Related GAO Products on the Results of the 

2000 Census and Lessons Learned for a More 

Effective Census in 2010

2000 Census: Refinements to Full Count Review Program Could Improve 

Future Data Quality. GAO-02-562. Washington, D.C.: July 3, 2002.

2000 Census: Coverage Evaluation Matching Implemented As Planned, 

but Census Bureau Should Evaluate Lessons Learned. GAO-02-297. 

Washington, D.C.: March 14, 2002.

2000 Census: Best Practices and Lessons Learned for a More CostEffective Nonresponse Follow-Up. GAO-02-196. Washington, D.C.:

February 11, 2002.

2000 Census: Coverage Evaluation Interviewing Overcame Challenges, 

but Further Research Needed. GAO-02-26. Washington, D.C.: December 31, 

2001.

2000 Census: Analysis of Fiscal Year 2000 Budget and Internal Control 

Weaknesses at the U.S. Census Bureau. GAO-02-30. Washington, D.C.: 

December 28, 2001.

2000 Census: Significant Increase in Cost Per Housing Unit Compared 

to 1990 Census. GAO-02-31. Washington, D.C.: December 11, 2001.

2000 Census: Better Productivity Data Needed for Future Planning and 

Budgeting. GAO-02-4. Washington, D.C.: October 4, 2001.

2000 Census: Review of Partnership Program Highlights Best Practices 

for Future Operations. GAO-01-579. Washington, D.C.: August 20, 2001.

Decennial Censuses: Historical Data on Enumerator Productivity Are 

Limited. GAO-01-208R. Washington, D.C.: January 5, 2001.

2000 Census: Information on Short- and Long-Form Response Rates.

GAO/GGD-00-127R. Washington, D.C.: June 7, 2000.

(450102)
GAO’s Mission The General Accounting Office, the investigative arm of Congress, exists to 

support Congress in meeting its constitutional responsibilities and to help improve 

the performance and accountability of the federal government for the American 

people. GAO examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of accountability, 

integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail this 

list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO 

Mailing Lists” under “Order GAO Products” heading.

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check 

or money order should be made out to the Superintendent of Documents. GAO 

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single 

address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Service Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00