Report to the Chairman, Committee on 

Oversight and Government Reform, 

House of Representatives 

United States Government Accountability Office

GAO 

April 2008 2010 CENSUS 

Bureau Needs to 

Specify How It Will 

Assess Coverage 

Follow-up Techniques 

and When It Will 

Produce Coverage 

Measurement Results 

GAO-08-414 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

April 2008

2010 CENSUS

Bureau Needs to Specify How It Will Assess

Coverage Follow-up Techniques and When It Will 

Highlights of GAO-08-414, a report to the Produce Coverage Measurement Results

Chairman, Committee on Oversight and 

Government Reform, House of 

Representatives 

An accurate decennial census relies 

on finding and counting people—

only once—in the right place and

collecting complete and correct 

information. GAO reviewed the 

Department of Commerce’s U.S. 

Census Bureau’s (Bureau) plans to

(1) improve the census’s

accuracy—or coverage—with its

coverage follow-up operation, (2)

assess the accuracy of the census 

through its census coverage

measurement program, and (3)

evaluate these efforts after 2010.

GAO reviewed documentation and 

interviewed Bureau officials about 

the Bureau’s coverage follow-up 

and measurement plans. GAO 

compared plans for measuring 

coverage in 2000 and 2010 and

interviewed 15 experts about 

changes to the plans. 

What GAO Recommends

GAO recommends that the

Secretary of Commerce direct the 

Bureau to specify (1) criteria it will 

use to assess techniques for 

identifying coverage follow-up 

cases; (2) when it will provide

coverage measurement estimates 

and how it will relate components

of coverage error to specific census 

operations to improve future

decennials; and (3) key decision 

points and plans for evaluating 

aspects of the 2010 coverage

follow-up and measurement efforts. 

In commenting on a draft of this 

report, Commerce had no 

substantive disagreements with our 

recommendations and will develop 

action plans for each one; and 

provided technical comments that

we incorporated as appropriate. 

The Bureau plans to expand its coverage follow-up operation to improve the 

accuracy of enumeration data collected in 2010. As in Census 2000, coverage 

follow-up is intended to resolve count discrepancies (when the number of 

household members reported on a completed questionnaire fails to match the 

number of persons for whom information is collected) and complete the 

enumeration of large households (more than six persons living in a housing 

unit). For 2010, the Bureau researched new techniques to increase the number 

of ways a person or household could be selected for coverage follow-up. 

These include (1) using coverage probes on the census form to identify cases

with potential under- or overcount problems, (2) using administrative records 

to identify households that include persons who may not have been counted, 

and (3) using computer matching to determine potential duplicate persons. 

However, it is not clear on what basis the Bureau will select or prioritize cases 

from this potentially expanded universe. The Bureau plans to further assess 

the techniques during the 2008 Dress Rehearsal, leaving little time to consider 

and implement improvements for the 2010 coverage follow-up operation. 

Because of the potential for the coverage follow-up workload to increase, it 

will be important for the Bureau to be clear on how it will decide which 

techniques to use and how it will select coverage follow-up cases to ensure

that the operation may best improve coverage. 

Although the Bureau’s overall design for measurement is similar in many

respects to the design for 2000, its measurement plan differs in important 

ways. The Bureau is planning to use a sample design similar to that used in 

2000 and still intends to produce net coverage error estimates. However, the 

Bureau does not plan to use measurement results to adjust the 2010 count of 

the nation’s population and is planning to make four significant changes: (1) 

conduct the measurement interview 4 months later than it did in 2000, (2) use 

a different method—logistic regression—to estimate net error, (3) use 

estimates of the components of coverage error in an effort to improve future 

decennials, and (4) expand computer matching and clerical matching to 

match persons in its postenumeration survey to the entire census. Still, the 

Bureau has not specified when it will produce estimates of net coverage error 

and the components of coverage error for the 2010 Census. The Bureau also 

has not clearly described how it will associate the components of coverage 

error to census operations in order to improve future decennials. Because of 

the Bureau’s changes to its coverage measurement program, it is important for 

the Bureau to solidify its plans and share them with stakeholders to ensure 

that there is an agreement on what the program will produce and when the 

Bureau will produce its results. 

The Bureau is in the early stages of planning how it will evaluate coverage 

follow-up and measurement after the 2010 Census. As part of this effort, the 

Bureau is considering recommendations of the National Academy of Sciences 

for the Bureau’s program of evaluations and experiments for Census 2010. The 

Bureau has not yet developed a timeline for completing its evaluation plans. 

To view the full product, including the scope 

and methodology, click on GAO-08-414. 

For more information, contact Mathew J. 

Scirè at (202) 512-6806 or sciremj@gao.gov. 
Page i GAO-08-414 Improving and Measuring Census Coverage 

Letter 1

Results in Brief 2 

Background 6 

Bureau Plans to Expand Coverage Follow-up to Improve Census 

2010 but Has Not Finalized Plans for Identifying Follow-up 

Cases 10 

Bureau Has Made Changes to Coverage Measurement for 2010 

Intended to Benefit the 2020 Census, but Key Aspects of 

Program Plan Remain Unclear 22 

Bureau Plans for Evaluating the 2010 Coverage Follow-up 

Operation and Census Coverage Measurement Program Are in 

Early Stages 33 

Conclusions 35 

Recommendations for Executive Action 37 

Agency Comments and Our Evaluation 38 

Appendix I Objectives, Scope, and Methodology 39 

Appendix II Comments from the Department of Commerce 43 

Appendix III GAO Contact and Staff Acknowledgments 50 

Figures 

Figure 1: Decennial Census Population Net Undercount Rates from 

the Bureau’s Demographic Analysis in Percentages, 1940-

2000 7 

Figure 2: Operation Flowchart for the Bureau’s 2008 Dress 

Rehearsal Coverage Follow-up Operation 12 

Figure 3: Undercount and Overcount Probes on Census 

Questionnaire for the 2008 Dress Rehearsal 15 

Figure 4: Timeline for Coverage Follow-up Interviews and Census 

Coverage Measurement Interviews Using Actual Dates for 

2000 and Estimated Dates for 2010 24 

Contents 
Page ii GAO-08-414 Improving and Measuring Census Coverage 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-08-414 Improving and Measuring Census Coverage 

April 15, 2008 

The Honorable Henry A. Waxman 

Chairman 

Committee on Oversight and Government Reform 

House of Representatives 

Dear Mr. Chairman: 

An accurate decennial census relies on finding and counting people—only 

once—in the right place and collecting complete and correct information. 

The Department of Commerce’s U.S. Census Bureau (Bureau) faces the 

daunting challenge of cost effectively counting a population that is 

growing steadily larger, more diverse, and according to the Bureau, 

increasingly difficult to find and reluctant to participate in the census. 

These challenges are made even more difficult by the Bureau’s April 3, 

2008 decision to redesign certain aspects of the decennial census. Two 

basic errors can affect the count of the population: omission of persons 

who should have been counted and erroneous enumerations of persons 

who should not have been counted, such as persons counted more than 

once or persons counted in the wrong geographic place. To ensure 

accuracy, the Bureau, among other operations, will conduct a coverage 

follow-up operation in 2010 to determine if additional persons might have 

been missed (undercounted) from already enumerated households, or if 

persons might have been counted in error (overcounted). Further, apart 

from the enumeration process, the Bureau will conduct a census coverage 

measurement program to assess the accuracy of the 2010 Census. 

The census is a critical national effort mandated by the Constitution. 

Census data are used to apportion seats in the Congress, redraw 

congressional districts, allocate billions of dollars in federal assistance to 

state and local governments, and for numerous other public and private 

sector purposes. The Bureau estimates that it has generally undercounted 

the total population in prior censuses. For the first time, in 2000, although 

estimates showed that it undercounted certain population groups, the 

Bureau estimates that it somewhat overcounted the total population. For 

2010, the Bureau plans to improve the accuracy of its population count by, 

among other things, expanding its coverage follow-up operation. Further, 

the Bureau and others questioned the accuracy of the data obtained from 

the Bureau’s coverage measurement program in 2000. As a result, for 2010, 

the Bureau plans to revise its methodology for determining coverage 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-08-414 Improving and Measuring Census Coverage 

error—the extent to which persons are under- or overcounted in the 

enumeration process. As agreed, we determined (1) how the Bureau plans 

to improve coverage through its coverage follow-up operation; (2) how the 

Bureau’s census coverage measurement plans for assessing the accuracy 

of the 2010 Census compare with efforts in 2000 and what the potential 

impact will be of those changes, including the Bureau’s plans for using its 

measurement program results to adjust population counts; and (3) how 

the Bureau plans to evaluate the coverage follow-up and measurement 

efforts after the 2010 Census. 

To meet these objectives, we reviewed information about the Bureau’s 

coverage follow-up and coverage measurement plans and interviewed 

Bureau officials. We reviewed relevant evaluations from Census 2000 and 

studies conducted by the Bureau and others, including the National 

Academy of Sciences. Further, we interviewed 15 experts about the 

Bureau’s coverage measurement methodology and plans. We identified 

these experts based on published reports as well as recommendations 

from the Bureau and the National Academy of Sciences. (For a list of 

experts we interviewed, see app. I.) We conducted this performance audit 

from January 2007 to March 2008 in accordance with generally accepted 

government auditing standards. Those standards require that we plan and 

perform the audit to obtain sufficient, appropriate evidence to provide a 

reasonable basis for our findings and conclusions based on our audit 

objectives. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit objectives. 

The Bureau plans to improve coverage by expanding the techniques it uses 

to identify and select cases for follow-up. However, it is not clear on what 

basis the Bureau will decide which techniques to use to identify potential 

cases, and which of these cases it will select or prioritize for coverage 

follow-up. As in Census 2000, coverage follow-up is intended to resolve 

count discrepancies (when the number of household members reported on 

a completed questionnaire fails to match the number of persons for whom 

information is collected) and complete the enumeration of large 

households (more than six persons living in a housing unit). The Bureau 

plans to include all such count discrepancy cases and all large households 

in its coverage follow-up universe in 2010. To improve coverage for 2010, 

the Bureau researched new ways to identify a more comprehensive set of 

cases for follow-up. Specifically, to increase the number of ways a person 

or household could be selected for follow-up, the Bureau tested three new 

coverage follow-up techniques: (1) using coverage probes on the census 

form mailed to households and during the nonresponse follow-up 

Results in Brief 
Page 3 GAO-08-414 Improving and Measuring Census Coverage 

interviews to identify instances where persons may be under- or 

overcounted; (2) using administrative records to identify households that 

include persons who may not have been counted; and (3) using computer 

matching to determine potential duplicate persons. The Bureau is 

uncertain how many cases for follow-up are likely to be found based on 

these new techniques. Nonetheless, the number of follow-up cases could 

be very large and very costly to resolve. Based on its experience with the 

2008 Dress Rehearsal, the Bureau expects to decide what techniques it will 

use for identifying and selecting cases during the 2010 Census coverage 

follow-up operation and how many cases it can afford to select for followup. Bureau officials explained that they intend to consider cost and the 

percentage of cases resulting in a change relative to the cost—known as 

yield—but the Bureau has not fully specified how it will make decisions 

about expanding the coverage follow-up universe or how it will select or 

prioritize cases from those identified by the new techniques. Using results 

from the 2006 Census Test, the Bureau has already decided that coverage 

follow-up will only be conducted by telephone rather than personal visits 

because the yield was higher for telephone follow-up. This decision, 

however, does not necessarily reflect trade-offs in the limitations of each 

technique used to select cases for follow-up. For example, coverage may 

not be improved for groups for which the Bureau does not have telephone 

numbers—the very wealthy or the very poor. Given that the Bureau 

expects to use the dress rehearsal to learn more about the expanded 

selection techniques, and that the dress rehearsal has been delayed by a 

month, the Bureau will have little time to complete its final plans for 

coverage follow-up for the 2010 Census. Because of the potential for the 

coverage follow-up workload to expand, it will be important for the 

Bureau to be clear on how it will decide which techniques to use and cases 

to select to ensure that coverage follow-up may best improve coverage. 

Although the Bureau’s plan for coverage measurement in 2010 is similar in 

many ways to its measurement design for 2000 and previous coverage 

evaluation programs, its plan differs in important ways, including its 

overall objectives, timing, statistical techniques used, and use of computer 

matching. Nonetheless, much of what the Bureau will do is yet to be 

decided, including most notably, when the Bureau will produce estimates 

of net coverage error and components of coverage error, and how it will 

meet one of its objectives for coverage measurement in 2010—improving 

future census operations. One important aspect of the Bureau’s plans for 

2010—specification of the logistic regression model—cannot be known 
Page 4 GAO-08-414 Improving and Measuring Census Coverage 

until after the Bureau collects the 2010 coverage measurement data. 

Similarities between 2000 and 2010 include the Bureau’s sample design for 

coverage measurement1

 and that the Bureau once again intends to use a 

dual systems approach to produce net coverage error estimates—or the 

difference between the coverage measurement program’s estimate of the 

total population and the decennial count of the total population.2

 However, 

several of the Bureau’s objectives for the 2010 coverage measurement 

program differ. For 2010, the Bureau places greater focus on improving 

future decennials. In contrast, for 2000, the Bureau designed coverage 

measurement to consider adjusting the count of the nation’s population. 

The Bureau does not plan to use coverage measurement results to adjust 

the 2010 count of the nation’s population because it believes that with a 

sufficiently correct count, adjustment would introduce as much or more 

error than it was designed to correct. The Bureau’s measurement program 

in 2010 differs in other significant ways from that in 2000, for example, (1) 

starting the coverage measurement interview about 4 months later, to 

permit expansion of the Bureau’s efforts to conduct the coverage followup operation;3

 (2) estimating net coverage error using logistic regression, 

which is a standard statistical modeling technique; (3) estimating the 

components of net coverage error to better understand the details of 

erroneous enumerations and omissions; and (4) expanding computer 

matching and clerical matching to match persons in the coverage 

measurement program’s postenumeration survey to the entire census to 

resolve or identify duplicates.4

 Bureau officials said that the coverage 

measurement interview will begin later because they expect to expand the 

coverage follow-up operation, and that to maintain independence and 

 1

The Bureau is planning to use a 300,000 national housing unit sample. 

2

The dual systems approach uses a postenumeration survey to produce an estimate of the 

true population, which is then used to produce an estimate of the net coverage error. The 

net error reflects the error remaining after omissions and erroneous enumerations offset 

one another. See GAO, 1990 Census Adjustment: Estimating Census Accuracy—A 

Complex Task, GAO/GGD-91-42 (Washington, D.C.: Mar. 11, 1991). 

3

Although the interviewing will start 4 months later compared to the Census 2000 telephone 

phase, person interviewing for the 2010 Census ends 1-1/2 months later than in 2000. 

4

The Bureau conducts a postenumeration survey, independent of the census, in a sample of 

block clusters. This sample is known as the P-sample and estimates the number of persons 

missed by the census. A postenumeration survey consists of two samples: the P-sample and 

the E-sample. P-sample persons are matched not only to the E-sample, but also to the 

entire census. E-sample persons are matched to the entire census to identify and resolve 

duplicates. E-sample housing units are matched to census enumerations in the block 

cluster and surrounding ring of blocks. 
Page 5 GAO-08-414 Improving and Measuring Census Coverage 

minimize possible contamination between enumeration and measurement, 

the Bureau did not want to overlap coverage follow-up and coverage 

measurement. Experts we interviewed were generally concerned about 

the contamination issue but also viewed the time delay as a very 

significant challenge because respondent recall5

 is vital to conducting the 

measurement program. The Bureau has simulated the use of its new 

method of estimating net error—logistic regression—by using data from 

2000 and found it to work. The Bureau will be unable to fully specify the 

model it will ultimately use until after the data collection phase of its 

coverage measurement program. One aspect of its plans that the Bureau 

should be able to specify now is when it will produce net coverage error 

estimates and components of coverage error estimates for 2010. Further, 

though the Bureau has preliminary plans—using data table shells—to 

associate components of coverage error to census operations, the Bureau 

has not clearly explained how it plans to meet its objective of improving 

future decennials. For the most part, experts we interviewed, after they 

reviewed our summary of the Bureau’s coverage measurement plans, 

stated that the Bureau had improved its research model and had taken 

significant and constructive steps in attempting to address key issues for 

coverage error, but several experts noted that more detailed planning is 

needed. Because of the Bureau’s changes to its coverage measurement 

program, it is important for the Bureau to make its plans clear and share 

them with stakeholders to ensure that there is an agreement on what the 

program will produce and when the Bureau will produce its measurement 

results. 

The Bureau’s plans for evaluating its coverage follow-up operation and 

census coverage measurement program for the 2010 Census are in early 

stages of development. In terms of evaluation, the Bureau suggested 

research on various aspects of coverage follow-up and measurement and 

contracted with the National Academy of Sciences to recommend 

proposed research for the 2010 Census. The academy’s final report is due 

by September 2009, but an interim report was issued December 2007. The 

interim report contained recommendations for the Bureau’s program of 

evaluations and experiments for the 2010 Census. Bureau officials have no 

timeline yet for making decisions about the evaluations of the Bureau’s 

coverage follow-up operation or census coverage measurement program 

 5

The coverage measurement interview depends upon the respondents’ ability to recall the 

composition of their households on Census Day. The longer the period of time between 

Census Day and the day that the measurement interview is conducted, the harder it is for 

respondents to recall membership of their households on Census Day. 
Page 6 GAO-08-414 Improving and Measuring Census Coverage 

following the 2010 Census, and they noted that to determine which 

recommendations to implement, the Bureau will consider costs and 

staffing needs. 

We are recommending that the Secretary of Commerce direct the Bureau 

to provide Congress and other census stakeholders with more specific 

plans on conducting coverage follow-up and census coverage 

measurement and evaluating the results of those programs for the 2010 

Census. To improve program operations, we recommend that the Bureau 

specify the criteria it will use to assess the techniques for identifying cases 

for coverage follow-up in the 2008 Dress Rehearsal and the timeline it 

plans to follow that will ensure that the results of the assessment may be 

used in planning for 2010. To enhance transparency and oversight for 

improving coverage measurement for the 2010 Census, we recommend 

that the Bureau describe when it will provide estimates of net coverage 

error and components of coverage error and how it plans to relate 

components of coverage error to census operations in order to improve 

future decennials. Finally, we recommend that the Bureau provide key 

decision points and plans for evaluating aspects of the 2010 Census 

coverage follow-up operation and census coverage measurement program. 

The Secretary of Commerce provided written comments on a draft of this 

report. The comments are reprinted in their entirety in appendix II and we 

have incorporated in the report, as appropriate, the Department of 

Commerce’s technical comments. Commerce had no substantive 

disagreements with our recommendations and stated that it will develop 

formal action plans for each recommendation. While Commerce believes a 

timeline for producing estimates of net coverage error and components of 

coverage error is certainly a reasonable expectation, Commerce noted that 

it is unclear as to how it can at this time provide any specifics on how 

coverage results might improve future censuses before the coverage data 

are obtained and analyzed. We recognize that the Bureau has developed 

table shells that associate components of coverage error to census 

operations, but these table shells and plans for how the Bureau will 

conduct its analyses have not been shared with the Congress and other 

stakeholders. In addition, it is not clear from current Bureau plans how it 

will go about obtaining the data required to complete the tables. 

Historically, the census has been affected by undercounts—those who 

were missed in the decennial count. Further, certain groups may be 

undercounted more than others—such as babies, minorities, and renters 

who move often. Seeking to obtain an accurate count has been a concern 

Background 
Page 7 GAO-08-414 Improving and Measuring Census Coverage 

since the first census in 1790. Concern about undercounting the 

population continued through the decades. In the 1940s, demographers 

began to obtain a more thorough understanding of the scope and nature of 

the undercount. For example, the selective service registration of October 

1940 showed 2.8 percent more men than the census count. According to 

the Bureau, operations and programs designed to improve coverage have 

resulted in the total undercount declining in all but one decade since the 

1940s. These measures of coverage are based on demographic analysis, 

which compares the census count to birth and death certificates and other 

administrative data (see fig. 1). 

Figure 1: Decennial Census Population Net Undercount Rates from the Bureau’s 

Demographic Analysis in Percentages, 1940-2000 

In contrast to the long-standing use of demographic analysis to estimate 

census undercount, modern coverage measurement began with the 1980 

Census, when the Bureau compared decennial figures to the results of an 

independent sample survey of the population. In using statistical methods 

such as these, the Bureau began to generate detailed measures of the 

differences among undercounts of particular ethnic, racial, and other 

groups. In 1990, the Bureau relied on its Post-Enumeration Survey to verify 

the data it collected through the 1990 Census. For this effort, the Bureau 

interviewed a sample of households several months after the 1990 Census, 

and compared the results to census questionnaires to determine if each 

sampled person was correctly counted, missed, or double counted in the 

0

1

2

3

4

5

6

1940 1950 1960 1970 1980 1990 2000 

Percentage

Source: U.S. Census Bureau.

5.4

4.1

3.1

2.7

1.2

1.6

0.1
Page 8 GAO-08-414 Improving and Measuring Census Coverage 

census. Using the definition of those missed minus those double counted, 

the Bureau estimated whether there was a net undercount or net 

overcount. For the 1990 Census, the Bureau estimated a net undercount of 

about 4 million people. During Census 2000, for the first time, the Bureau’s 

coverage measurement program measured a net population overcount. To 

estimate the accuracy of the 2000 Census, the Bureau conducted its 

Accuracy and Coverage Evaluation, expecting it could be used to adjust 

the Census 2000 results for all nonapportionment purposes if it improved 

the census data. The original March 2001 estimates of the Accuracy and 

Coverage Evaluation, which was an independent sample survey designed 

to estimate the number of people who were under- and overcounted in the 

census, a problem that the Bureau refers to as coverage error, became 

available in time to correct census files. However, in a revised March 2003 

evaluation, the Bureau was unable to conclude that the adjusted data were 

more accurate and determined that the 2000 Census tabulations would not 

be adjusted for any purpose. The decision not to adjust was consistent 

with decisions made during the 1990 decennial, when the Department of 

Commerce decided, because of other problems, not to use the 

postenumeration survey to adjust the decennial count. 

During this decade, the Bureau has conducted a testing and evaluation 

program to assess certain activities that it believes can improve the 

accuracy of the census. In preparing for the 2010 Census, the Bureau has 

conducted two field tests—in 2004, the Bureau tested in the Queens 

Borough of New York City and southwest Georgia and, in 2006, tested at 

the Cheyenne River Reservation in South Dakota and parts of Austin, 

Texas, in Travis County. In 2003 and 2005, the Bureau also conducted 

national surveys to test various aspects of changes planned for 2010. We 

reported earlier about the importance of these earlier assessments leading 

to a design that would be sufficiently mature so that the dress rehearsal 

for the 2010 Census demonstrates the feasibility of the various operations 

and technologies planned for the decennial under conditions that are as 

close as possible to the actual census.6

 The Bureau is currently conducting 

its dress rehearsal, one of the final planning activities for the decennial. 

Census Day for the 2008 Dress Rehearsal activities in sites located in 

 6

For example, see GAO, 2010 Census: Cost and Design Issues Need to Be Addressed Soon, 

GAO-04-37 (Washington, D.C.: Jan. 15, 2004); 2010 Census: Basic Design Has Potential, 

but Remaining Challenges Need Prompt Resolution, GAO-05-9 (Washington, D.C.: Jan. 12, 

2005); and 2010 Census: Census Bureau Should Refine Recruiting and Hiring Efforts 

and Enhance Training of Temporary Field Staff, GAO-07-361 (Washington, D.C.: Apr. 27, 

2007). 
Page 9 GAO-08-414 Improving and Measuring Census Coverage 

North Carolina and California will be May 1, 2008, a delay of 1 month from 

the Bureau’s initial plan. 

The first 2010 Census operation, building an accurate address list, overlaps 

with dress rehearsal activities and has also already begun.7

 For the 2010 

Census, the majority of the housing units in the country will receive paper 

census questionnaires delivered either by mail or by census field workers 

before April 1, 2010. Those households that do not return their 

questionnaires will be contacted by census field workers during the 

nonresponse follow-up operation to determine the number of people living 

in the housing unit on Census Day and other information. The Bureau also 

uses other techniques for counting persons. For example, people in group 

quarters (such as college dormitories, homeless shelters, and nursing 

homes) will be enumerated in separate operations at the facilities by 

enumerators who will then leave forms listing the names of the people 

living or staying there and then return to pick up the completed forms and, 

if necessary, conduct follow-up interviews. 

For the 2010 Census, the Bureau has tested and planned revisions to the 

census form with the goal of improving the census’ coverage. The 

revisions are intended to improve methods for ensuring that persons are 

counted in the correct place and only once.8

 These changes include 

revising residency rules, simplifying the instructions provided to 

respondents on residency, and including questions—called coverage 

probes—about persons who may be missing or incorrectly included in the 

response. For Census 2000, the Bureau relied on 31 residence rules to 

instruct respondents on who to include on the mail-back form. The Bureau 

revised these 31 rules to 1 rule with 13 broad living situations for 2010. For 

2010, the Bureau is also planning on using two coverage probes, one 

asking about potentially missed people (undercounted) and one asking 

about potentially duplicated people (overcounted), on the census form. 

 7

Starting in January 2007, the Bureau notified state and local governments that it would 

seek their help in developing a complete address file through the Local Update of Census 

Addresses program. Address canvassing—a field operation to build a complete and 

accurate address list in which census field workers go door to door verifying and 

correcting addresses for all households and street features contained on decennial maps—

will begin in April 2009. For more information, see GAO, 2010 Census: Census Bureau Has 

Improved the Local Update of Census Addresses Program, but Challenges Remain, 

GAO-07-736 (Washington, D.C.: June 14, 2007). 

8

See also National Academy of Sciences, Once, and Only Once, and in the Right Place: 

Residence Rules in the Decennial Census (Washington, D.C.: 2006). 
Page 10 GAO-08-414 Improving and Measuring Census Coverage 

Although the Bureau used coverage probes in the nonresponse follow-up 

operation for 2000, it did not include probes in the 2000 Census mailed 

questionnaires as is now planned for 2010. The Bureau intends to use 

answers to these coverage probes and other techniques to determine if an 

enumeration is likely to be incorrect or incomplete and, therefore, if cases 

should be selected for follow-up. The Bureau uses the results of the 

coverage follow-up operation to correct the results of the initial 

enumeration. Coverage follow-up interviews with respondents determine 

if changes should be made to their household rosters as reported on their 

initial census returns. Specifically, these results lead to a correction of 

census information and counts, as coverage follow-up results are 

combined with the initial enumeration and are used as the official 

enumeration of the household. 

For Census 2010, the Bureau is planning to improve coverage by 

expanding its coverage follow-up operation to obtain more accurate 

enumeration data. To prepare for this, the Bureau has conducted tests of 

an expanded follow-up operation. As in Census 2000, coverage follow-up is 

intended to resolve count discrepancies and complete the enumeration of 

large households. For 2010, the Bureau also plans to use additional 

techniques to increase the number of ways a person or household could be 

selected for follow-up. However, the Bureau has not finalized plans for 

how to identify and select cases from the expanded follow-up universe. 

The expansion of selection criteria for coverage follow-up has the 

potential to dramatically increase the workload for this operation and, as a 

result, increase costs. While the Bureau plans to budget for coverage 

follow-up in order to achieve the best results or highest yield of changes to 

household rosters, it is unclear on what basis the Bureau will assess the 

additional techniques for identifying potential cases or the basis for 

selecting cases for follow-up using the expanded techniques. The Bureau 

plans to assess its experiences with coverage follow-up, including 

important interfaces within the Bureau and between the Bureau and its 

contractor, during the 2008 Dress Rehearsal in making further decisions 

on follow-up for 2010. For example, three separate groups within the 

Bureau have responsibility for selecting follow-up cases, but one of those 

groups eliminates overlap in cases before sending them to the contractor 

for telephone follow-up. Recent decisions to delay the dress rehearsal, 

including follow-up activities, increase the risk that the results of 

additional testing of these techniques in 2008 will not be sufficiently timely 

or complete to improve operations for 2010, and the Bureau has not stated 

when it will complete its assessment of coverage follow-up for the 2008 

Dress Rehearsal. 

Bureau Plans to 

Expand Coverage 

Follow-up to Improve 

Census 2010 but Has 

Not Finalized Plans 

for Identifying 

Follow-up Cases 
Page 11 GAO-08-414 Improving and Measuring Census Coverage 

The Bureau is planning to improve coverage by expanding its coverage 

follow-up operation. To meet this objective, the Bureau expects to rely on 

previously used techniques as well as techniques it is now developing to 

identify cases for follow-up. As in Census 2000, coverage follow-up for 

2010 will include 

• resolving count discrepancies, when the number of household 

members reported on a completed questionnaire fails to match the 

number of persons for whom information is collected, and 

• completing the enumeration of large households, in which there are 

more than six persons living in a housing unit. 

For 2010, the Bureau also plans to use additional techniques for selecting a 

person or household for follow-up. The techniques planned include 

• using questions—called coverage probes—on the census form itself 

and in the nonresponse follow-up interviews to identify where persons 

may be omitted or counted more than once, 

• using administrative records to identify persons who may be omitted, 

and 

• using computer matching from the universe of census returns to 

identify persons who may have been counted more than once. 

Although the Bureau has not finalized coverage follow-up plans for the 

2010 Census, in June 2007, the Bureau developed a work flow and 

narrative for the 2008 Dress Rehearsal coverage follow-up operation. This 

work flow shows the process for collecting enumeration data, selecting 

coverage follow-up cases, and conducting follow-up (see fig. 2). Once the 

Bureau selects a case for follow-up, it plans to work through its Decennial 

Response Integration System (DRIS) contractor, which will conduct 

coverage follow-up interviews by telephone. The telephone interviews 

determine if changes should be made to the household rosters as reported 

on the initial census returns. The questions in the follow-up interview are 

designed to identify if persons were omitted or counted in error because 

they should have been counted at a different address or not at all—for 

example, because a person had been born after or died before Census Day. 

Bureau Plans to Expand 

Techniques for Identifying 

Coverage Follow-up Cases 
Page 12 GAO-08-414 Improving and Measuring Census Coverage 

Figure 2: Operation Flowchart for the Bureau’s 2008 Dress Rehearsal Coverage Follow-up Operation 

As shown in figure 2, enumeration data will be captured by DRIS. Also, in 

selecting cases for follow-up, the Bureau will identify any overlap in cases 

represented by large households, count discrepancies, coverage probes, 

administrative records review, and computer matching. While the Bureau 

has tested new techniques for coverage follow-up, it has not yet tested 

Collection process

Selection process

Telephone process

Paper

forms

Source: GAO analysis of U.S. Census Bureau data.

Bureau’s Decennial Systems and Processing 

Office eliminates overlap to determine coverage 

follow-up universe

Bureau’s Decennial Systems and 

Processing Office identifies coverage 

follow-up cases by flagging for:

DRIS sends

completed response 

data to the Bureau

Large households

Count discrepencies

Bureau’s Data Integration 

Division identifies cases for 

coverage follow-up

by conducting:

Administrative 

records matching

Affirmative answers to 

coverage probes

DRIS assigns identification numbers and

prepares case list for telephony 

DRIS conducts

telephone calls with 

coverage follow-up cases

DRIS normalizes

telephony 

information

DRIS normalizes

incoming information

Bureau’s Decennial Statistical 

Studies Division identifies

cases for coverage follow-up 

by conducting: 

National computer 

matching
Page 13 GAO-08-414 Improving and Measuring Census Coverage 

these steps with the new processing interfaces within the Bureau—

Decennial Systems and Processing Office, Decennial Statistical Studies 

Division, and Data Integration Division—and between the Bureau and 

DRIS. During the 2008 Dress Rehearsal, from May 27 to August 9, 2008, the 

Bureau will not only be assessing various aspects of the expanded followup, but it will also be testing for the first time the DRIS integration of data 

as well as the Bureau’s process for identifying overlap cases. After the 

Bureau selects cases for follow-up, the DRIS contractor assigns an 

identification number to each case and oversees telephone interviews. 

Each case may receive up to seven follow-up telephone calls, and the DRIS 

contract includes such aspects as on-demand automated call distribution, 

call routing, reporting, and quality assurance monitoring. 

For 2010, as in Census 2000, all count discrepancy cases identified on 

mailed paper census forms will be part of the coverage follow-up 

operation. The Bureau will check completed questionnaires for 

discrepancies between the number of persons reported as members of a 

household and the number of persons for whom census information was 

provided on the census forms. For example, questionnaires that fail edit 

checks might include cases where the respondent may indicate that five 

people were living in the housing unit on Census Day, but there is 

information for only two people. 

As in 2000, the Bureau is also aiming to improve coverage of people in 

housing units by completing the enumeration of large households, 

meaning those that indicate on their census forms that there are more than 

six persons in the housing unit and for which there was not enough room 

on the mailed questionnaire form for self-response. The questionnaire has 

space for only six household members to be fully listed. The nonresponse 

follow-up interview is not limited in the number of people that can be 

reported and all household members can be enumerated during 

nonresponse follow-up; therefore, no large household cases will need to 

be enumerated during the coverage follow-up operation as a result of the 

nonresponse follow-up interview. 

The Bureau plans to expand its universe for coverage follow-up in 2010 by 

including cases resulting from the use of coverage probes. For the dress 

rehearsal, the Bureau plans to contact anyone who affirmatively answers 

the undercount question (which is at the household level) and the 

overcount question (which is at the person level) (see fig. 3). If either or 

both coverage probes prove unproductive in the dress rehearsal, Bureau 

officials indicated that they might be eliminated from the follow-up 

operation. The coverage probes are included on the mail-back 

Resolving Count Discrepancies 

Enumerating Large Households 

Using Coverage Probes to 

Identify Potential Under- or 

Overcounting 
Page 14 GAO-08-414 Improving and Measuring Census Coverage 

questionnaire and among the questions asked during the nonresponse 

follow-up. During nonresponse follow-up, for example, if a person 

responds in the affirmative to the coverage undercount probe, the 

interviewer collects the name and demographic information for the added 

person. All names of individuals added to the household roster9

 through 

the undercount probe during nonresponse follow-up will be sent to the 

coverage follow-up operation for additional follow-up. 

 9

A household that completes a coverage follow-up interview could either finish the 

interview with the same roster of household members listed at the beginning of the 

interview or make some change to the roster. A roster change occurs when a household 

adds at least one person to the original roster, deletes at least one person from the original 

roster, or both. 
Page 15 GAO-08-414 Improving and Measuring Census Coverage 

Figure 3: Undercount and Overcount Probes on Census Questionnaire for the 2008 Dress Rehearsal 

As shown in figure 3, for the 2008 Dress Rehearsal, the undercount probe 

asks if there are any additional people staying at the residence on Census 

Day that were not included earlier—children, such as newborn babies or 

foster children; relatives, such as adult children, cousins, or in-laws; 

nonrelatives, such as roommates or live-in baby sitters; or people staying 

at the residence temporarily. The overcount probe for the 2008 Dress 

Rehearsal asks if persons sometimes lived or stayed elsewhere—such as 

while attending college, while in the military, while staying at a seasonal or 

second residence, during a child custody arrangement, while in jail or 

prison, while in a nursing home, or for another reason. The overcount 

probe is asked of the first six household members on the mail-back 

questionnaire and of all persons in the nonresponse follow-up interview. 

Evaluations of the coverage follow-up operation for the Bureau’s 2004, 

2005, and 2006 tests examined the performance of both the undercount 

Undercount probe

Overcount probe

Source: GAO presentation of U.S. Census Bureau information.
Page 16 GAO-08-414 Improving and Measuring Census Coverage 

and overcount probes. In the Bureau’s 2004 test, the overcount probe was 

found to be more effective in finding households with erroneous 

enumerations than the undercount probe was in finding persons omitted 

from the count. In this test several problems were identified with the 

undercount probe, most notably a low rate of added residents to the 

household roster. Results from the test showed that of the 3,919 

respondents at the Georgia test site who answered the undercount probe 

affirmatively, only 53 persons were added to the household roster. The 

Bureau noted that the low rate of added persons was “perplexing.” Other 

problems included difficulties with the interviewer reading the probe as 

worded and indications that use of the probe may have actually increased 

the number of count discrepancy cases. 

In 2005, the Bureau tested two different versions of both the undercount 

and overcount probes, and changes to both probes were adopted for the 

2006 Census Test. During the 2006 test, the Bureau found that the 

overcount probe “performed as expected” and generated a higher rate of 

deleted persons from the census test roster for both the mail-back 

questionnaire and nonresponse follow-up operation than other new 

techniques tested to identify coverage follow-up cases. Although the 

Bureau designed the undercount probe to add persons, the undercount 

probe resulted in more deleted persons than added persons.10 The Bureau 

noted that over half of the deletions stemmed from cases in which the 

respondent either answered “yes” to both coverage probes or the 

respondent answered “yes” to the undercount probe and had a count 

discrepancy. Indeed, in a separate analysis that examined the rate of 

overlap among coverage follow-up cases—meaning households were 

flagged for coverage follow-up through more than one technique—the 

Bureau found that the highest significant rate of overlap was among cases 

where the respondent answered “yes” to both coverage probes. 

Additionally, there were some operational difficulties using the 

undercount probe during the nonresponse follow-up operation in the 2006 

Census Test.11 For example, the Bureau found that although persons 

 10Of the 4,191 completed interviews, at least one person was added to 360 households while 

at least one person was deleted from 531 households. 

11During the nonresponse follow-up operation, if persons reply affirmatively to the 

undercount probe, the census field workers collect their names and demographic 

information. Then, these persons appear on the coverage follow-up roster and the coverage 

follow-up interview confirms the Census Day residency for these persons. 
Page 17 GAO-08-414 Improving and Measuring Census Coverage 

answered the undercount probe affirmatively (1,234 households) during 

nonresponse follow-up, very few persons (8) were added to the coverage 

follow-up roster.12 The Bureau referred to this number as “surprisingly 

low,” because if a household replies “yes” to the undercount probe, the 

Bureau expected to see a person added to the roster for coverage followup. In its evaluation of coverage follow-up during the 2006 Census Test, 

the Bureau speculated that there could have been operational error during 

nonresponse follow-up that led to this discrepancy. Furthermore, of those 

8 persons who were added and appeared in coverage follow-up, only 3 

could be confirmed as residents during follow-up, leading the Bureau to 

conclude that the undercount question added persons incorrectly more 

than half of the time after nonresponse follow-up. Although the Bureau 

has made some slight modifications to the response options for the 

overcount probe it plans to use in the dress rehearsal, it plans to use the 

same undercount probe for the dress rehearsal as it did for the 2006 test. 

For the dress rehearsal, the Bureau plans one wave of matching with 

administrative records to identify duplicate or suspected incorrect data. 

For administrative matching, the Bureau will use the Enhanced Statistical 

Administrative Records System, which is a Bureau-wide resource that 

collects certain data from the Internal Revenue Service, the Social Security 

Administration,13 the Selective Service, the Department of Housing and 

Urban Development, as well as other federal agencies. The Bureau will 

match census returns against records from these sources using various 

algorithms to associate persons with housing units to identify households 

with potential undercounting.14 The Bureau plans to match names on 

census questionnaires with names in the administrative database—if a 

household is found to have more persons in the administrative records 

than on the census form, it would be a candidate for coverage follow-up. 

The Bureau first tested selecting cases for follow-up through comparison 

with administrative records in 2005. For the 2006 Census Test, all of the 

cases with matches from the administrative records search had personal 

 12While only 8 persons were included in the coverage follow-up roster after nonresponse 

follow-up, the coverage follow-up added an additional 69 persons from households that 

were initially enumerated through nonresponse follow-up. 

13Bureau officials said that its administrative records system uses a unique identifier in 

place of the Social Security number. Social Security numbers are only used for the purpose 

of validating the Bureau’s administrative records system. 

14If the records do not match, the Bureau does not inform the other agencies because of 

issues related to census confidentiality. 

Using Administrative Records 

to Identify Potentially Omitted 

Persons 
Page 18 GAO-08-414 Improving and Measuring Census Coverage 

visits; however, Bureau officials noted that personal visits were conducted 

because staff had sufficient time, given the smaller geographic area of a 

test site. Personal visits will not be part of the national 2010 coverage 

follow-up operation. The 2008 Dress Rehearsal marks the first time that 

cases selected through administrative record matching will be tested in the 

coverage follow-up telephone operation. 

For the dress rehearsal, the Bureau will conduct several waves of 

computer matching for its unduplication process. In 2000, the Bureau had 

difficulty with duplicate enumerations and did not have a planned 

operation for dealing with the problem; officials explained that in 2000 

there was an “ad hoc” plan that they developed “on the fly” for identifying 

duplicates. However, Bureau officials noted that one of the lessons 

learned from 2000 was that the Bureau needed to expand the search for 

duplicates by computer matching. Thus, to improve the 2010 Census, along 

with the overcount probe specifically designed to try and find duplicates, 

the Bureau will also match census returns against the universe of all other 

census returns to identify duplicates nationwide; for the dress rehearsal, 

the matching will be conducted within the test site, rather than on a 

national scale. While the Bureau has been evaluating and testing this 

procedure, the limited likelihood of duplication of individuals that will 

occur within census test sites of limited geographic size, as compared to 

the actual national decennial, limits the Bureau’s ability to forecast how 

the nationwide matching and unduplication system will affect workload 

for coverage follow-up.15

Although the Bureau has not finalized its coverage follow-up techniques, it 

plans to apply a computer-based record linkage system to the data 

collection files during the 2010 Census to match files against themselves to 

find potential duplicates based on housing unit and person information. 

Those households would then be selected for follow-up. For example, 

erroneous enumerations are created when duplicated people are counted 

correctly in one housing unit and incorrectly in another housing unit. 

Duplicated housing units, which occur when multiple addresses of the 

 15In the 2004 Census Test, the Bureau was successful in identifying duplicates using person 

matching, but it discovered operational complications with trying to resolve person and 

housing situations within the same operation. Basically, the Bureau concluded that with 

such a complicated interview, it was too much to ask of the interviewers that they 

understand the distinction between person and housing duplication. A major component of 

the 2006 Census Test was to find a way to determine before follow-up whether person 

duplication was resulting from housing-level issues or person-level issues. 

Using Computer Matching to 

Identify Persons Who May 

Have Been Duplicated 
Page 19 GAO-08-414 Improving and Measuring Census Coverage 

same housing unit are identified during census operations, are removed. 

Further, in contrast to Census 2000, the addresses that could be included 

in the census in both the housing unit and group quarter enumerations can 

no longer cause duplication because, according to the Department of 

Commerce, the Bureau now has one list for both housing units and group 

quarters. Consequently, according to Bureau officials, the Bureau faces the 

challenge of identifying the types of duplicates that can be most easily 

identified and addressed, which will result in the most cost-effective yield. 

The Bureau’s 2006 test of computer matching found that potential 

duplicates generally were not deleted. Specifically, in its evaluation of 

computer matching during the 2006 Census Test, the Bureau found that 93 

percent of respondents marked as possible duplicates of persons counted 

outside the census block were neither deleted nor linked to a deleted 

match. The Bureau considered this result disappointing. It noted that the 

coverage follow-up interviews did not resolve the cases very well because 

follow-up interviews were not completed for 24 percent of the cases and, 

in 61 percent of the cases, respondents did not confirm during the followup interview that they lived or stayed somewhere else. The interviewer 

cannot provide information about the suspected duplicate during the 

follow-up interview because of concerns related to confidentiality; the 

interview is limited in its potential for success in that the respondent has 

to volunteer information about the suspected duplicate case. 

Consequently, the Bureau is not only challenged in identifying suspected 

duplicate cases, but in actually resolving the cases as well. 

Bureau officials explain that for the unduplication process, the Bureau will 

assign a match score and identify links among suspected duplicates. The 

Bureau plans to develop a match score threshold that will vary by level of 

geographic distance between matches. The Bureau will then assess the 

quality of the match based on the match score and conduct further 

research on the cases exceeding this threshold. For example, once the 

Bureau identifies potential duplicates, it determines whether the potential 

duplication is a potential housing-level problem (duplicated persons 

identified within a census block) or person-level problem (more long 

distance-identified duplicates nationally). If it is a housing-level problem, 

then the Bureau would consider using personal visits by field staff from 

another of its operations, the field verification operation, to verify housing 

unit additions. Further, according to the Department of Commerce, field 

verification may be undertaken to determine if housing units should be 

deleted from the list. If it is a person-level problem, the Bureau would 

consider a coverage follow-up operation telephone interview to determine 

if the person is counted twice. 
Page 20 GAO-08-414 Improving and Measuring Census Coverage 

The Bureau is developing a number of techniques for selecting cases for 

follow-up and plans to expand the coverage follow-up operation overall. 

However, it has not determined which techniques it will use, what 

thresholds it will use in selecting cases for follow-up, or the resources it 

expects to devote to the coverage follow-up operation. The most pressing 

question remaining is how the Bureau will prioritize its expanded universe 

of follow-up cases identified through coverage probes, administrative 

record matching, and the unduplication process. Currently, households 

that display count discrepancies, are large, reply in the affirmative to 

coverage probes, are flagged through comparisons with administrative 

records, or are flagged as potential duplicates are all eligible for follow-up. 

Although there may be a large number of eligible cases for coverage 

follow-up, the Bureau is unsure how many cases it can financially afford 

for follow-up, and discussions are still under way within the Bureau about 

what units will be sent to the coverage follow-up operation. The Bureau is 

reviewing data from tests—such as the add and delete rates within each 

type of case—to aid in determining what types of techniques will be most 

productive in terms of identifying overcounted and missed persons. 

The Bureau plans to rely on additional data from its assessment of the 

coverage follow-up operation in the 2008 Dress Rehearsal in making 

decisions about which techniques, including coverage probes, 

administrative record matching, and the unduplication process, it plans to 

use for identifying and selecting cases for coverage follow-up in 2010. 

Dress rehearsal plans for coverage follow-up should mirror the plans 

intended for the decennial because the 2008 Dress Rehearsal will be the 

Bureau’s last opportunity to assess the various procedures and systems for 

2010 under as close to census-like conditions as possible. However, the 

expanded coverage follow-up operation conducted during the dress 

rehearsal will not necessarily be what the Bureau implements in 2010 

because the Bureau plans to use the dress rehearsal to assess key aspects 

of a likely coverage follow-up operation, including (1) the potential 

coverage follow-up workload and how it may be distributed, (2) the extent 

to which the coverage follow-up operation improves census coverage 

through such factors as the percentage of added residents and deleted 

residents, and (3) problems administering the follow-up interview and 

managing work. The assessment will also address problems with the 

operational work flow and interfaces and provide recommendations to fix 

or improve these problems. While the Bureau’s expectation is that it will 

have the necessary data from the dress rehearsal to inform the final plans 

for the 2010 coverage follow-up operation, Bureau officials have noted that 

the delayed dress rehearsal (a delay in the Census Day from April 1 to May 

Bureau Will Rely on 

Assessment of Dress 

Rehearsal to Complete 

Coverage Follow-up Plans 

for the 2010 Census 
Page 21 GAO-08-414 Improving and Measuring Census Coverage 

1, 2008) may make it more difficult to incorporate the findings from the 

dress rehearsal into the Bureau’s plans for the 2010 Census. 

Bureau officials have stated in several forums that resource constraints 

are a factor in the final decision on the scope of follow-up in 2010. The 

expansion of selection criteria has the potential to dramatically increase 

the workload for coverage follow-up in 2010 and, as a result, increase 

costs. Bureau officials have cautioned that it is not necessarily costeffective for the Bureau to follow up on the entire universe eligible to be 

included in the coverage follow-up operation. Thus, the Bureau will be 

using results—such as the percentage of cases resulting in a change 

relative to the cost or yield—from the dress rehearsal to decide which 

techniques to use and how many cases it can afford to include for followup. 

Evaluations of the impact of the criteria the Bureau will use in judging 

what techniques to use for identifying cases for follow-up or the 

thresholds for selecting specific cases for follow-up will be critical to 

determining the final plan for the coverage follow-up operation. The 

Bureau has made one judgment about how it will proceed for 2010 as a 

result of its 2006 test. Specifically, in its evaluation of the 2006 Census 

Test, the Bureau calculated that the cost was $155 per roster change for 

coverage follow-up cases done by telephone versus $766 per roster change 

for cases done by personal visits. Consequently, the Bureau concluded 

that it should conduct follow-up interviews by telephone. This decision, 

based partly on the percentage of cases with a change and partly on cost, 

represents two criteria. 

Effective program management, including assessment of risks,16 requires 

that agencies identify information that is critical to achieving important 

program objectives.17 In this case, the Bureau should clearly state the basis 

for deciding which techniques to use or the number of cases it will include 

for coverage follow-up. Because the Bureau has not specified these 

criteria, it is not clear what criteria—other than roster change and cost—

the Bureau will consider. For example, the Bureau noted in its decision to 

conduct coverage follow-up by telephone that not adding a personal visit 

 16See GAO, Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (Washington, D.C.: Nov. 1999). 

17See GAO, Internal Control Management and Evaluation Tool, GAO-01-1008G 

(Washington, D.C.: Aug. 2001). 
Page 22 GAO-08-414 Improving and Measuring Census Coverage 

component to the operation would limit its ability to reduce coverage 

errors for households it could not reach by telephone. Bureau research 

indicated that households for which it did not have telephone numbers 

tended to be very wealthy or very poor. Thus, coverage may not be 

improved for these demographic groups through telephone follow-ups 

alone. 

The Bureau’s overall design of its census coverage measurement program 

for the 2010 Census is similar in many respects to its measurement design 

for 2000 but differs in some important ways. The objectives, timing, and 

some techniques to be used for 2010 differ from those used in 2000; 

however, the Bureau has not made clear when it will produce estimates of 

net coverage error and components of coverage error, nor does it make 

clear how it will meet one of its objectives for coverage measurement in 

2010—improving future census operations. In 2000, the Bureau focused on 

estimating net coverage error with the possibility of producing adjusted 

numbers at the census block level. In contrast, for 2010, while still 

intending to produce net coverage error estimates at higher levels of 

geography—the national and state levels—the Bureau has refocused its 

efforts in order to use its measurement program to help improve 

enumeration methodologies for the future. Importantly, the Bureau does 

not plan for the measurement program to prepare adjusted numbers for 

the 2010 Census. In addition to starting coverage measurement interviews 

4 months later than it did in 2000, the Bureau has also made several other 

important changes to its measurement program. For example, to better 

estimate net coverage error, the Bureau plans to use logistic regression for 

the first time. The Bureau also plans to assess the association of 

components of coverage error to various census operations. Finally, to 

better measure coverage, the Bureau plans to better identify duplicates by 

expanding its computer and clerical matching. Expert statisticians and 

methodologists we interviewed generally agreed that the Bureau’s changes 

for its 2010 measurement program are an improvement from the 2000 

coverage measurement program; however, several experts noted that 

more detailed planning is needed. 

For the 2010 coverage measurement program, the Bureau expects to 

include multiple operations. First, the Bureau selects its sample of census 

block clusters to be included in the coverage measurement program. Using 

the sample, the Bureau collects information on the housing units in the 

census blocks to develop an address list independent from that used in the 

census enumeration. The Bureau then uses this address list to draw a 

sample of housing units to include in the person interview operation. 

Bureau Has Made 

Changes to Coverage 

Measurement for 2010 

Intended to Benefit 

the 2020 Census, but 

Key Aspects of 

Program Plan Remain 

Unclear 
Page 23 GAO-08-414 Improving and Measuring Census Coverage 

During this operation, Bureau field staff interview—in person—residents 

of these housing units and collect demographic, Interview Day and Census 

Day residency information of all persons living at the housing unit. 

According to the Department of Commerce, additional information will be 

collected in 2010 to identify alternate addresses where persons on the 

coverage sample roster may have been counted on Census Day and this 

information will be used in person-based computer and clerical matching 

operations. Next, the Bureau compares data collected during the person 

interviews with all census enumerations. Where the Bureau is unable to 

make a match, the Bureau follows up by sending interviewers back to the 

housing unit to reinterview the person or persons in question. During the 

final operation, the Bureau produces estimates of coverage error for 

persons and housing units. For 2010, the Bureau plans to provide 

estimates of net coverage error at the national level by race/ethnicity, 

age/sex, and tenure—the status of an occupied housing unit as either 

owner occupied or renter occupied. The Bureau will also produce 

estimates of net coverage error for states. 

The Bureau’s 2010 coverage measurement program sample design is 

similar to that used for measurement in 2000. As in 2000, the Bureau plans 

to use a 300,000 national housing unit sample. The methods are basically 

consistent with the 2000 coverage measurement program and are expected 

to yield similar levels of reliability, although slightly different methods are 

being used to determine state sampling allocation. The objective for 2010 

is to have the sample allocated proportionally across states and the 

District of Columbia, with a minimum of 1,800 housing units in each state. 

Moreover, the Bureau is using a differential within state sample allocation 

in order to allow for oversampling in areas as needed. Notably, according 

to Bureau officials, it is possible that that the sampling numbers could still 

change depending on budget or operational constraints. Most experts we 

spoke with thought that the Bureau’s sample design was appropriate for 

its objectives. 

In 2000, the timeline for coverage measurement overlapped with the 

Bureau’s coverage follow-up efforts, though it was conducted 

independently from the census enumeration; however, measurement and 

coverage follow-up are planned to be conducted sequentially for the 2010 

Census. Specifically, in 2010, the census coverage measurement interview 

will begin after the coverage follow-up operation is completed (see fig. 4). 

According to the Bureau’s 2010 Census Operations and Systems Plan, 

dated August 31, 2007, the Bureau expects to conduct the person 

2010 Sample Design for 

Census Coverage 

Measurement Is Similar to 

That for 2000 

Census Coverage 

Measurement Interviews 

Will Begin Later, and 

Estimates Will Take 

Longer to Complete in 

2010 
Page 24 GAO-08-414 Improving and Measuring Census Coverage 

interviews for coverage measurement from August 20 through October 16, 

2010. In 2000, the Bureau conducted the coverage measurement person 

interviews from April 24 through September 11, 2000. 

Figure 4: Timeline for Coverage Follow-up Interviews and Census Coverage 

Measurement Interviews Using Actual Dates for 2000 and Estimated Dates for 2010 

According to the Bureau, the expansion of the coverage follow-up 

operation contributed to delaying the measurement program in order to 

maintain data independence and reduce the likelihood of data 

contamination resulting from the overlap of follow-up and measurement 

field operations.18 In past coverage measurement programs, the person 

interview phase overlapped with coverage follow-up conducted as part of 

the census enumeration. In 2010, the Bureau plans to delay the start of the 

person interview phase for coverage measurement to minimize the risk of 

data contamination. While many experts agreed that delaying the coverage 

measurement interviews will reduce the risk of data contamination, some 

experts also highlighted that the delay may also negatively affect the data 

quality because it could contribute to recall bias. Recall bias, as it pertains 

to coverage measurement, means that a person incorrectly recalls where 

he or she or other household members resided on Census Day and thus 

 18Data contamination occurs when a person’s inclusion or exclusion from one census 

operation affects the probability of his or her inclusion in a second independent census 

operation. 

2000

2010

Coverage measurement

Coverage follow-up

Coverage follow-up Coverage measurement

April May June July

April May June July

August

May June July August

August

September

September October

Source: GAO analysis of U.S. Census Bureau data.
Page 25 GAO-08-414 Improving and Measuring Census Coverage 

biases the coverage measurement data. Two experts also noted that 

delaying coverage measurement interviews could also affect quality of 

data on movers. Generally, experts said that decreasing the amount of 

time between the coverage follow-up interview and the coverage 

measurement interview would benefit the coverage measurement data. 

However, a few experts who were also familiar with the issue stated that 

they were unsure of the magnitude of contamination and, therefore, did 

not know if the Bureau should decrease the time. Several of these experts 

argued that the Bureau could make a better case by using results from its 

contamination studies to determine if the delay is warranted. In 2006, the 

Bureau conducted a contamination study using the results of the 2006 

Census Test to determine if the coverage measurement person interviews 

caused a difference in responses to the coverage follow-up interviews.19

Although the Bureau stated that the number of sample cases involved in 

the study was small and that the result should be viewed with caution, it 

concluded that there might be some small impact on coverage follow-up 

results if the measurement interview is conducted at the same time as the 

coverage follow-up interview. 

For 2010, the Bureau also expects to complete its coverage error estimates 

later than was the case for the Bureau’s initial 2000 estimates. In 2000, the 

Bureau produced initial estimates in March 2001; however, after evaluating 

the 2001 estimates, the Bureau released revised estimates in March 2003. 

According to Bureau officials, the Bureau has preliminary plans to 

complete coverage measurement estimation20 for the 2010 Census 

sometime after October 201121—6 months after the Bureau provides 

population counts for redistricting. Bureau officials have stated that the 

decision to delay measurement estimation will allow for more time to 

adequately evaluate the 2010 measurement data and estimates, noting that 

the Bureau did not have sufficient time for analysis in 2000. 

 19The experts were not aware of these results because the documents were for internal 

Bureau use only. 

20According to Bureau officials, the dress rehearsal timeline is the best current indicator of 

the timeline for the 2010 Census. However, Bureau officials have told us that coverage 

measurement estimation will not be done for the 2008 Dress Rehearsal. In addition, the 

Bureau will not include its coverage measurement housing unit field operation, housing 

matching operation, or person follow-up. The Bureau is also considering dropping person 

matching from the dress rehearsal. 

21The exact date is yet to be determined. 
Page 26 GAO-08-414 Improving and Measuring Census Coverage 

The Bureau’s 2010 coverage measurement objectives have changed from 

those of Census 2000. In 2000, the Bureau focused on estimating net 

coverage error with the possibility of adjusting the census count at the 

census block level of geography. Though the Bureau still intends to 

produce net error estimates for 2010—at the state level and higher—it has 

refocused the coverage measurement program on improving future 

enumeration methodologies; consequently, it does not plan to use the 

results of its coverage measurement program to improve or adjust the 

count of the nation’s population in 2010. For 2010, the Bureau’s goals for 

measuring coverage accuracy include the following aspects: 

• The Bureau plans to use a dual systems approach to estimate net 

coverage error by using logistic regression instead of the 

poststratification technique used in 2000. The Bureau believes its move 

to logistic regression will also allow more flexibility because it can 

incorporate a greater number and type of factors or predictors in the 

model. 

• The Bureau plans to improve future census operations by associating 

estimates of components of coverage error, including erroneous 

enumerations and omissions, to census operations. 

• To better resolve and identify duplicate enumerations, a major source 

of error in previous decennials and coverage measurement programs, 

the Bureau plans to expand computer and clerical matching to match 

persons in its postenumeration survey to the entire census from the 

more limited matching done in 2000. 

The Bureau plans to conduct a census coverage measurement program for 

the 2010 Census, but Bureau officials said that they did not know when the 

Bureau would produce detailed documentation describing its plans. The 

Bureau has developed preliminary plans for associating components of 

coverage error to census operations, but these plans also are not finalized. 

According to the Bureau, it has no plans to use the results of its coverage 

measurement program to improve or adjust the 2010 Census count. In 

response to a May 12, 2003 letter from Representative William Lacy Clay 

asking about decennial plans, the Bureau explained that with a sufficiently 

correct count, adjustment would introduce as much or more error than it 

was designed to correct. If the Bureau were asked by the Congress to 

provide adjusted numbers, Bureau officials commented that it would be 

unable to quickly or easily do so because the current coverage 

measurement program’s timeline produces estimates after October 2011, 

and the Bureau has not designed the software to produce estimates at 

lower geographic levels. Consequently, according to the Bureau, it cannot 

Objectives for the 2010 

Census Coverage 

Measurement Differ from 

Those for 2000, but How 

the Bureau Will Meet New 

Objectives Is Unclear 

Bureau Does Not Plan to Adjust 

the 2010 Census in Time for 

Redistricting 
Page 27 GAO-08-414 Improving and Measuring Census Coverage 

prepare adjusted numbers in time to meet the legal deadline for producing 

count data for redistricting, which is April 1, 2011. Furthermore, the 

Bureau is not planning to provide net error estimates below the state level; 

such estimates are necessary for redistricting. 

Some of the interviewed experts agreed that producing adjusted numbers 

would be challenging; however, several also said that the Bureau could 

still provide some adjusted numbers if asked to do so. Regarding the 

challenges, two experts stated that adjusted numbers could be produced 

at a higher geographic level, the national level, but not accurately at lower 

geographic levels, the state and local levels, without appropriate changes 

to the current design and methodology. For example, one expert explained 

that the Bureau would need to use poststratification to produce estimates 

at the state and local levels. Additionally, several experts noted that the 

2010 coverage measurement timeline poses a challenge for producing 

adjusted numbers by the April 1, 2011, redistricting deadline. 

The Bureau plans to use logistic regression to improve its estimation of net 

coverage error. Of the experts familiar with logistic regression, the 

majority thinks that the methodology is an improvement; however, a few 

experts question how the Bureau will explain a more complex method to 

its stakeholders. Previously, for the 2000 coverage measurement, the 

Bureau used a poststratification22 method. However, Bureau officials said 

that poststratification was limited because of the small sample sizes of 

many of the strata. They explained that logistic regression permits greater 

flexibility in dealing with small sample sizes because it allows for a greater 

number and type of factors or predictors in the model.23 By definition, 

poststratification is limited because it only allows for a small number of 

factors with defined categories to be included. In addition to greater 

flexibility, logistic regression can potentially reduce errors in estimation. 

According to Bureau officials, to verify estimates from logistic regression, 

the Bureau is also considering using some poststratification. Logistic 

 22Poststratification entailed calculating dual systems estimates separately for different 

demographic and geographic groups. Dual systems estimation requires that a 

postenumeration survey be conducted. It produces an estimate of the true population, 

which is then used to produce an estimate of the net coverage error. In the 2000 coverage 

measurement program, over 400 strata or groups were examined. 

23For instance, logistic regression allows for the use of continuous variables like age, 

instead of only the categorical variables used in poststratification. Additionally, unlike 

poststratification, logistic regression is able to include or omit interactions between 

variables. 

Bureau Plans to Use a Different 

Statistical Method for 

Estimating Net Error but Has 

Not Fully Specified Its Plans 
Page 28 GAO-08-414 Improving and Measuring Census Coverage 

regression provides greater flexibility in dealing with small samples, but it 

may be more difficult methodologically to use logistic regression to 

produce state-level estimates by race/ethnicity, age/sex, and tenure or to 

estimate lower than the state level. Some experts argued for continuing to 

produce estimates using poststratification to enable direct comparisons of 

error estimates to prior decennials and to verify that logistic regression is 

working. 

To enhance transparency and oversight for improving coverage 

measurement for the 2010 Census, the Bureau should provide the 

Congress with timely and accurate information to allow monitoring of 

agency activities—specifically, timely and accurate information about its 

plans for coverage measurement.24 The Bureau plans to refine its 

estimation methods for 2010 by using logistic regression to estimate net 

coverage error, but it does not have an operational or working plan that 

outlines what resources—such as the amount of time that it will take to 

run the data set(s)—will be used to complete this operation. Further, 

despite conducting some exploratory logistic regression work using the 

2006 test data, as well as simulating its estimation model for logistic 

regression using decennial data from 2000 and finding workable models, 

the Bureau has not documented what approaches or processes it intends 

to apply in the further development of the regression model or models it 

plans to use. In addition, the Bureau does not have a written plan detailing 

how it will explain to stakeholders and Bureau data users the rationale for 

moving from poststratification to logistic regression and what effect this 

change may have on data interpretation—particularly as it relates to 

comparisons with past census data. Experts we interviewed also noted 

some challenges, including communicating the new method to users of 

census data and stakeholders, having an adequate number of staff to 

complete the analysis, and designing the best models—such as choosing 

the variables to include. 

The Bureau plans to estimate the components of coverage error for 2010 in 

order to associate the errors to census operations to improve future 

decennials; in 2000, the coverage measurement program focused on net 

coverage error, not components of coverage error. Although the Bureau 

will once again estimate net coverage error, Bureau documentation shows 

that estimating the components of coverage error is now a higher priority 

for the Bureau. Coverage error comprises two types of errors: errors of 

 24See GAO-01-1008G. 

Bureau’s New Focus on 

Estimating Components of 

Coverage Error Is Intended to 

Benefit Future Decennials, but 

Plans Are Incomplete 
Page 29 GAO-08-414 Improving and Measuring Census Coverage 

omission and errors from a person or housing unit mistakenly enumerated 

(erroneous enumerations). The impact of omissions and erroneous 

enumerations are combined to determine the net error. 

During Census 2000, the Bureau estimated the net coverage error of the 

census. However, net error can disguise significant count problems—the 

Bureau could have a large overcount in some areas and large undercount 

elsewhere, and it might balance out to a small net error nationwide. In 

addition, the coverage measurement definitions of erroneous 

enumerations used in 2000 were not as precise as those that will be used in 

2010. For example, the 2000 coverage measurement program did not 

require Bureau officials to ascertain where a person should have been 

counted. Instead, some enumerations were treated as omissions or 

erroneous enumerations in an area where they should have been counted. 

According to the Department of Commerce, because of the Bureau’s new 

goal to measure the components of coverage error, the Bureau will use a 

definition for component error estimation that is not as strict as that used 

for net error. For 2010, erroneous enumerations for the coverage 

measurement program are defined to include the following: duplicate 

enumerations, persons born after or who die before Census Day, or 

persons who are not residents of a housing unit in the United States on 

Census Day. 

Though the Bureau recognizes the potential benefits of estimating the 

components of coverage error to improve future decennials, it has not 

fully developed its plans for doing so in 2010. Providing the Congress and 

other key stakeholders with information about how the Bureau plans to 

use the 2010 coverage measurement program to improve future decennials 

would enhance transparency and oversight of coverage measurement.25

For Census 2000, the Bureau did not associate the estimates of erroneous 

enumerations and omissions to specific census operations. For the 2010 

Census, the Bureau has not finalized plans to evaluate the quality of its 

estimates of components of coverage error (i.e., estimates of variance) and 

has only recently determined the level of geography for which estimates 

will be made. Moreover, although the Bureau has developed table shells 

that associate components of coverage error to census operations, the 

Bureau has not clearly explained how it plans to implement operations 

and data collection strategies that will support the analyses and lead to the 

completion of the tables and convert those analyses into plans to improve 

 25See GAO-01-1008G. 
Page 30 GAO-08-414 Improving and Measuring Census Coverage 

the census operation for 2020. Importantly, Bureau officials stated that it 

will be difficult to link estimates of the components of coverage error to 

specific census operations. Instead, they characterize the process of 

associating the components of error to census operations as an 

exploratory effort. A Bureau official explained that associating errors of 

omission to operations will be most challenging because omitting persons 

from the census count could be caused by errors in multiple operations. 

For example, a person could have been omitted because of a bad address 

or an incorrect enumeration. It may not be possible to determine which 

operation contributed to the omission. In contrast, she said that it may be 

more feasible to associate errors of erroneous enumerations to individual 

census operations because there is more that they know about the 

enumerations. 

In associating errors to census operations, a Bureau official stated that the 

Bureau may use a modeling approach to identify those factors with the 

highest probability and greatest significance in predicting specific 

erroneous enumeration errors. Additionally, Bureau officials noted that 

how the Bureau associates components of coverage error to census 

operations cannot be determined without first determining the final 

models for estimating the components of coverage error for the 2010 

Census. These models cannot be specified without the actual 2010 Census 

data and therefore will be specified after data collection. Finally, as 

recommended by the National Academy of Sciences, Bureau officials 

acknowledged that there is ongoing experimental research involving the 

use of a master trace sample database in associating the components of 

coverage error to census operations. This database is intended to help the 

Bureau trace response and operational data through various stages of the 

2010 Census—such as address listing and data collection. The National 

Academy of Sciences notes that such a database should allow the Bureau 

to retain a full history of relevant census operations for a designated 

sample of households in census blocks. The Bureau should then be able to 

use the database to investigate the source of a variety of potential 

deficiencies that can arise in a complicated undertaking like the decennial 

census. More specifically, the database could be used to determine what 

percentage of census omissions are from partially enumerated households 

and what percentage of omissions are from the merged administrative 

records database. According to a Bureau official, the Bureau plans to 

begin developing the database in 2008. However, the official noted several 

limitations with such a database. For example, the database consists of a 

sample of the U.S. population, as opposed to the whole census population. 

Still, Bureau officials noted that improving the master trace sample 
Page 31 GAO-08-414 Improving and Measuring Census Coverage 

database in 2010 is important, citing the example of using the database for 

complex cluster analyses that link to coverage measurement. 

While most experts we contacted agreed that estimating the components 

of coverage error will likely help to improve future decennials, some 

experts questioned the Bureau’s ability to link the actual errors to specific 

census operations. For example, one expert said that it “will be practically 

impossible to project errors to operations.” Additionally, some experts 

discussed other challenges that the Bureau may face in accurately 

estimating the components of coverage error, including distinguishing 

between different types of erroneous enumerations and omissions as well 

as resolving duplicates and accurately matching the data. Still, some 

experts did say that it may be possible for the Bureau to link certain 

erroneous enumerations to actual operations. Because of the Bureau’s 

new objective to estimate components of coverage error and relate them 

to census operations, it is important for the Bureau to solidify its plans and 

share them with stakeholders to ensure that there is an understanding on 

what the program will produce in order to improve future decennials. 

Similar to the matching planned to expand the coverage follow-up 

operation, the Bureau’s plans for its 2010 coverage measurement program 

include expanding the matching process in order to better resolve and 

identify duplicate enumerations—a major source of error in the 2000 

Census and coverage measurement program. Some of the experts we 

contacted are confident that the plans will be an improvement. Shortly 

after the census enumeration is completed, a postenumeration survey will 

be conducted in a sample of block clusters. This re-enumeration sample—

known as the P-sample—is used with the E-sample (the actual census 

enumeration in those sample areas) to derive a corrected population 

estimate.26 For 2010, the Bureau will also use computer matching to 

compare the P-sample with all census enumerations, with the objective of 

identifying and resolving duplicate enumerations. This is an expansion 

from 2000, when the P-sample was matched against enumerations from 

sample blocks and surrounding blocks only, although housing units are 

matched to census housing in the search area. As in 2000, the Bureau will 

 26According to the Department of Commerce, the P-sample is used to estimate the match 

rate, while the E-sample is used to estimate the correct enumeration rate. Both of these 

samples are needed to derive a corrected population estimate. Matching is performed to 

the census, not the E-sample. 

Bureau Plans Expanded 

Matching for Resolving and 

Identifying Duplicate 

Enumerations 
Page 32 GAO-08-414 Improving and Measuring Census Coverage 

also use clerical matching to do additional matching, but will also do 

nationwide matching.27 

Despite the Bureau’s effort to improve its matching process and minimize 

the number of duplicates in 2010, the Bureau does not yet have complete 

plans for the 2008 Dress Rehearsal or for the 2010 Census to help 

accomplish this goal. Bureau officials noted that materials are being 

developed for certain matching procedures, such as clerical matching. 

Further, Bureau officials said that budgetary constraints may also affect 

the matching operations and changes may occur before or following the 

2008 Dress Rehearsal. For example, the Bureau may not conduct person 

matching during the dress rehearsal. 

Though a few experts cited the challenges the Bureau will continue to face 

in resolving duplicate enumerations, several also noted that the expanded 

matching process should help to reduce the number of duplicates 

considerably. An expert explained that by expanding the geography 

covered in the Bureau’s matching operations, it will be easier to identify 

duplicates as well as households that may have been overlooked in 

previous coverage measurement matching operations because the search 

area was limited to respective census blocks and a few nearby 

surrounding geographic areas. Still, he also said that the expanded 

matching will be an improvement only if implemented well. 

 27Bureau officials have noted that clerical matching procedures being planned for 2010 are 

largely comparable to those used for coverage measurement in the 2000 Census although, 

with additional information being collected in 2010, the Bureau will have to account for 

duplicate searching at alternate respondent-provided addresses collected in the person 

interviews and person follow-up interviews. 
Page 33 GAO-08-414 Improving and Measuring Census Coverage 

The Bureau is in the early stages of developing plans for evaluating its 

2010 Census coverage follow-up operation and census coverage 

measurement program. To assist in this effort, the Bureau has contracted 

with the National Academy of Sciences to recommend proposed research 

for the 2010 Census. The academy’s final report is due in September 2009.28

In its December 2007 interim report, the academy recommended that the 

Bureau conduct certain experiments and evaluations, such as assessing 

the use of administrative records and coverage probes for identifying 

cases for follow-up.29 Bureau officials have also suggested research on 

various aspects of coverage follow-up and measurement for 2010. Officials 

said that the Bureau will consider costs and staffing needs in deciding 

what it will evaluate. However, the Bureau has not yet developed a 

timetable for making decisions about evaluating the 2010 coverage followup operation or census coverage measurement program. 

For the most recent decennial censuses, the Bureau has carried out 

experiments and evaluations. A census experiment usually involves field 

data collection during the census in which alternatives to current census 

processes are used and assessed for a subset of the population. An 

evaluation is usually an analysis conducted on data collected as part of the 

decennial process to determine whether individual steps in the census 

operated as expected. The Bureau plans to work with internal and 

external stakeholders on defining and developing its program for 

experiments and evaluations for the 2010 Census, with design work and 

implementation starting in 2008 and continuing through 2011. The final set 

of activities would include analysis, documentation, and presentation of 

the research, and these activities would start in 2009 and be completed by 

the end of the fiscal year 2013. 

According to the National Academy of Sciences’ December 2007 interim 

report, its purpose was to reduce the possible subjects for census research 

from a list of over 50 research topics. The academy reviewed the Bureau’s 

initial list of research topics to help reduce the possible subjects for 

census experimentation, and offered broad advice in its interim report on 

 28The National Academy of Sciences has convened a panel of experts to review the 

Bureau’s program of research, evaluation, and experimentation for the 2008 Dress 

Rehearsal and the 2010 Census. The panel will consider priorities for evaluation in the 2010 

Census. The panel will conduct its work over a 3-year period, from September 2006 to 

September 2009. 

29National Academy of Sciences, Experimentation and Evaluation Plans for the 2010 

Census: Interim Report (Washington, D.C.: Dec. 7, 2007). 

Bureau Plans for 

Evaluating the 2010 

Coverage Follow-up 

Operation and Census 

Coverage 

Measurement 

Program Are in Early 

Stages 
Page 34 GAO-08-414 Improving and Measuring Census Coverage 

plans for evaluations of the 2010 Census. For this smaller number of 

research topics, the academy recommended several experiments and 

evaluations relating to coverage improvement and measurement. For 

example, the academy recommended that the Bureau assess the use of 

administrative records in the event that the Bureau opts to review 

administrative records to identify potential omissions. It also 

recommended that the Bureau include one or more alternate questionnaire 

experiments, such as an examination of the usefulness of the residence 

rules and concepts on the census questionnaire. Further, the academy 

recommended a possible experiment on comparing telephone interviews 

to personal interviews for the coverage follow-up operation because a 

telephone-only operation would follow up for only those households that 

provide a contact telephone number. However, at this point, the Bureau 

has not yet made definitive decisions about what it will evaluate and when 

it will make these decisions. 

In its interim report, the National Academy of Sciences suggested that the 

Bureau design a master trace sample database—a tool for investigating the 

source of potential deficiencies that can arise in such a complicated 

undertaking as the decennial census—that would help assess the benefits 

of the coverage follow-up interview. This database could link person, 

household, and housing unit characteristics; census processes; and the 

presence or absence of census component coverage error. The academy 

suggested that the use of this database would better identify the sources of 

high rates of census component coverage error, including erroneous 

enumerations, enumerations in the wrong place, duplications, and 

omissions. The academy also suggested that the Bureau use a reverse 

record check for coverage measurement. It noted that the reverse record 

check is an alternative method for estimating the completeness of census 

coverage of the population, which may have advantages over the methods 

of dual systems estimation and demographic analysis that have been used 

for this purpose to date. This procedure has not been used previously to 

evaluate census data, mainly because of the 10-year period between 

censuses, which complicates the need to trace people’s addresses from 

one census to the next. However, the Bureau now uses the American 

Community Survey, a monthly survey of 250,000 housing units to collect 

and provide long-form data on an annual basis. A reverse check would 

facilitate comparing decennial enumerations to information collected 

more frequently during the American Community Survey. 

The National Academy of Sciences believes that it will be very important 

for the Bureau to finalize specific designs of its experiments by summer 

2008 to meet the planning needs of the census. Because the data needed to 
Page 35 GAO-08-414 Improving and Measuring Census Coverage 

support census evaluations are typically output files from the census itself, 

the exact structure of individual evaluations is not yet as time-sensitive as 

the experiments. However, the academy notes that some early planning for 

evaluations is critical so that the necessary data extracts can be prepared 

and retained, especially because much of the data collection in 2010 will 

be carried out by contractors, and thus data retention requirements need 

to be arranged with contractors as early as possible. It also suggests that 

the Bureau increase its in-house expertise in experimental design for the 

census. Some experts we interviewed about the Bureau’s plans for 

coverage measurement also noted that the Bureau should plan its 

evaluations with more urgency. An expert noted that in 1990 and 2000, 

when the Bureau faced challenges with its adjustment process, there were 

fewer staff members available to work on evaluations; he said that perhaps 

this would not be a problem for the 2010 Census because the Bureau did 

not plan to adjust the census count. Further, a few experts suggested that 

the Bureau should ask outside researchers, in addition to the National 

Academy of Sciences, for assistance and feedback in its evaluation 

planning and development. A few experts also suggested that the Bureau 

should consider developing more qualitative evaluations, such as studies 

to look deeper into why certain populations, such as the homeless, are 

omitted from the census at greater rates. 

With only 2 years remaining before Census Day, the Bureau faces 

formidable challenges in successfully implementing and completing the 

2010 Census. These challenges are even more formidable given the 

Bureau’s recent decision to redesign certain aspects of the 2010 Census, 

and make more critical the need for the Bureau to have in place specific 

and concrete plans for ensuring and measuring the accuracy of the 

decennial census. To accurately count everyone residing in the United 

States, the Bureau must overcome significant challenges due to the 

nation’s increasing diversity, as well as, according to the Bureau, a 

population increasingly reluctant to participate in surveys. In response to 

these challenges, the Bureau plans to expand its coverage follow-up 

operation—a final quality assurance method intended to help accurately 

enumerate the population—by focusing on resolving count discrepancies, 

enumerating large households, and following up on households with 

potential enumeration problems identified through techniques the Bureau 

plans to further test during its dress rehearsal. As in prior decennials, the 

Bureau plans to measure the accuracy of the 2010 decennial through a 

coverage measurement program. The Bureau intends to focus its coverage 

measurement efforts in 2010 on learning more about the components of 

census error in order to improve operations during the next decennial—

Conclusions 
Page 36 GAO-08-414 Improving and Measuring Census Coverage 

2020—and future decennials. The Bureau does not plan to use its 2010 

coverage measurement program to adjust the count of the nation’s 

population. 

The Bureau has initiated efforts to improve the accuracy of the census 

count by expanding coverage follow-up and improving measurement; 

however, some areas bear enhanced monitoring, oversight, and 

management as the Bureau moves from its 2008 Dress Rehearsal to final 

plans for the 2010 Census. Providing internal and external stakeholders a 

full description of its plans for coverage follow-up and measurement will 

help the Bureau ensure that the plans for and expected outcomes of these 

important operations are understood not only by the Bureau but also by 

stakeholders. Importantly, many of the operational decisions for the 2010 

Census depend on the upcoming 2008 Dress Rehearsal. For example, the 

Bureau has not finalized its plans for 2010 regarding the scope of its 

coverage follow-up operation and will base final decisions on how to 

select cases for follow-up in part on its experiences in the dress rehearsal. 

Coverage follow-up improvements have been tested on a site-level scale 

but have not yet been tested with new processing interfaces between the 

Bureau and its contractor for collecting and disseminating information. 

The Bureau plans to expand coverage follow-up but has not clearly stated 

the basis it will use for deciding which techniques to use or the number of 

cases it will include for follow-up. The Bureau will rely on the dress 

rehearsal to assess the effectiveness of certain coverage follow-up 

techniques, such as the undercount probe. Existing analyses suggest that 

this probe is of limited use. Overall, given the uncertainty surrounding key 

aspects of the Bureau’s plans for coverage follow-up for 2010, as well as 

the delayed dress rehearsal, the content and magnitude of the 2010 

coverage follow-up bear continued scrutiny. 

Operational plans for coverage measurement lack specificity, and the 2010 

Census will be the first time that the Bureau can implement its expanded 

computer matching for both coverage follow-up and measurement on a 

national level. Importantly, although an announced purpose for revising 

the census coverage measurement program was to provide improvements 

for the 2020 Census and the Bureau is working toward associating 

components of error to specific operations, it has neither clearly explained 

how it will do this nor the expected dates for when its plans for this 

analysis will be complete. All of these unknown aspects increase the 

level(s) of risk of not meeting these important goals. By having plans and 

timelines in place, the Bureau increases the likelihood of completing its 

coverage measurement analyses in time to inform the planning for the 

2020 decennial. 
Page 37 GAO-08-414 Improving and Measuring Census Coverage 

Because the Bureau plans significant changes to its coverage follow-up 

and measurement activities, it should be planning well in advance of the 

decennial how it will evaluate and measure their success. Timely and 

informed oversight, monitoring, and management can promote the success 

of these evaluative activities. The Bureau’s program for evaluations and 

experiments has great potential to help improve the next decennial. The 

National Academy of Sciences has made interim recommendations on 

various ways that the Bureau could assess its decennial data collection 

activities. Among other things, the academy recommends using a master 

trace sample because such a sample would improve the Bureau’s efforts to 

relate census errors to census operations. We believe that such a master 

trace sample has the potential to help the Bureau better understand 

sources of error. Nonetheless, designing such a sample is a daunting task. 

The Bureau has begun to evaluate how to implement the academy’s 

recommendation. The academy’s interim report also suggested that the 

Bureau soon finalize specific designs of the experiments to meet its 

planning needs so that necessary data retention requirements could be 

arranged with contractors as early as possible. 

We are recommending that the Secretary of Commerce direct the Bureau 

to provide the Congress and other census stakeholders with more specific 

plans on conducting coverage follow-up, implementing coverage 

measurement, and evaluating the results of those programs for the 2010 

Census. Specifically, we recommend that the Bureau provide 

• the criteria it will use to assess the techniques for identifying cases for 

the coverage follow-up operation in the 2008 Dress Rehearsal and the 

timeline for completing this assessment to permit planning for 2010—

the timeline should specify dates for preliminary assessments, final 

assessments, and when operational changes might be made to the 2010 

Census; 

• its plans for conducting coverage measurement for the 2010 Census, 

which should include a description of when it will provide estimates of 

net coverage error and components of coverage error and how it plans 

to relate components of coverage error to census operations in order to 

improve future decennials; and 

• key decision points and plans for evaluating aspects of the 2010 Census 

coverage follow-up operation and census coverage measurement 

program. 

Recommendations for 

Executive Action 
Page 38 GAO-08-414 Improving and Measuring Census Coverage 

We provided a draft of this report to the Secretary of Commerce for his 

review and comment. Commerce had no substantive disagreements with 

our recommendations and stated that it will develop formal action plans 

for each recommendation. While Commerce believes a timeline for 

producing estimates of net coverage error and components of coverage 

error is certainly a reasonable expectation, Commerce noted that it is 

unclear as to how it can at this time provide any specifics on how 

coverage results might improve future censuses before the coverage data 

are obtained and analyzed. We recognize that the Bureau has developed 

table shells that associate components of coverage error to census 

operations, but these table shells and plans for how the Bureau will 

conduct its analyses have not been shared with the Congress and other 

stakeholders. In addition, it is not clear from current Bureau plans how it 

will go about obtaining the data required to complete the tables. 

As agreed with your office, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days after the 

report date. We will then send copies of this report to the Secretary of 

Commerce, the Department of Commerce’s Inspector General, the 

Director of the U.S. Census Bureau, and interested congressional 

committees. We will make copies available to others upon request. This 

report will also be available at no charge on our Web site at 

http://www.gao.gov. 

If you or your staff have any questions concerning this report, please 

contact me at (202) 512-6806 or sciremj@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on the 

last page of this report. GAO staff who made major contributions to this 

report are listed in appendix III. 

Sincerely yours, 

Mathew J. Scirè 

Director, Strategic Issues 

Agency Comments 

and Our Evaluation 
Appendix I: Objectives, Scope, and 

Methodology 

Page 39 GAO-08-414 Improving and Measuring Census Coverage 

To determine (1) how the U.S. Census Bureau (Bureau) plans to improve 

coverage through its coverage follow-up operation, (2) how the Bureau’s 

census coverage measurement plans for assessing the accuracy of the 2010 

Census compare with efforts in 2000, and (3) how the Bureau plans to 

evaluate the coverage follow-up and measurement efforts after the 2010 

Census, we requested and obtained source documents from the Bureau’s 

headquarters in Suitland, Maryland. We also interviewed Bureau officials 

in several of its divisions, including the Decennial Management Division 

and the Decennial Statistical Studies Division. Further, we reviewed 

evaluations regarding coverage follow-up and coverage measurement. 

In addition, for the second objective, related to the Bureau’s census 

coverage measurement program, we interviewed 15 experts about the 

Bureau’s coverage measurement methodology and plans. Prior to our 

interviews, we summarized the Bureau’s plans for coverage measurement 

and met with Bureau officials to obtain confirmation of our summary. We 

then identified experts based on published reports as well as 

recommendations from the Bureau and the National Academy of Sciences 

and contacted them about reviewing our summary of the Bureau’s 

measurement plans and interviewing them about the Bureau’s plans, 

including the extent to which the Bureau’s plans addressed concerns and 

limitations of previous coverage evaluation programs, the extent to which 

using logistic regression instead of poststratification would improve the 

estimation of net coverage error, the main challenges in estimating 

components of coverage error, and the implications of delaying the 

coverage measurement interviews. Fifteen experts responded 

affirmatively to our request for interviews and we subsequently 

interviewed the following persons: 

Dr. Barbara Bailar 

Statistical Consultant 

Washington, D.C. 

Dr. Thomas Belin 

Professor of Biostatistics, Psychiatry, and Biobehavioral Sciences 

Department of Biostatistics 

School of Public Health 

University of California, Los Angeles 

Los Angeles, California 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology 

Page 40 GAO-08-414 Improving and Measuring Census Coverage 

Dr. Michael Brick 

Senior Statistician 

Vice President, Director of Survey Methods 

Associate Director of the Statistical Staff at Westat 

Rockville, Maryland 

Dr. James Brown 

Senior Lecturer in Official Statistics 

School of the Social Sciences 

University of Southampton 

Southampton, United Kingdom 

Mr. Dave Dolson 

Director 

Social Survey Methods Division 

Statistics Canada 

Ottawa, Canada 

Dr. Stephen Fienberg 

Maurice Falk University Professor of Statistics and Social Science 

Department of Statistics 

Carnegie Mellon University 

Pittsburgh, Pennsylvania 

Dr. Tim Johnson 

Director 

Survey Research Laboratory 

University of Illinois at Chicago 

Chicago, Illinois 

Mr. Don Royce 

Director General 

Methodology Branch 

Statistics Canada 

Ottawa, Canada 

Dr. Duane Steffey 

Director 

Statistical and Data Sciences 

Exponet, Inc. 

Menlo Park, California 
Appendix I: Objectives, Scope, and 

Methodology 

Page 41 GAO-08-414 Improving and Measuring Census Coverage 

Dr. John Thompson 

Executive Vice President of Survey Operations 

National Opinion Research Center at the University of Chicago 

Chicago, Illinois 

Dr. Robert Tortora 

Chief Methodologist 

Gallup Washington 

Washington, D.C. 

Dr. Roger Tourangeau 

Director 

Joint Program in Survey Methodology 

University of Maryland 

College Park, Maryland 

and 

Acting Director 

Program in Survey Methodology 

University of Michigan 

Ann Arbor, Michigan 

Dr. Kenneth Wachter 

Professor of Demography and Statistics 

University of California at Berkeley 

Berkeley, California 

Dr. Martin Wells 

Director of Research 

Charles A. Alexander Professor of Statistical Sciences 

Cornell University 

Ithaca, New York 

Dr. Kirk Wolter 

Head of the Center for Excellence in Survey Research 

Professor in the Department of Statistics 

Senior Fellow at the National Opinion Research Center 

University of Chicago 

Chicago, Illinois 

We conducted this performance audit from January 2007 to March 2008 in 

accordance with generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 
Appendix I: Objectives, Scope, and 

Methodology 

Page 42 GAO-08-414 Improving and Measuring Census Coverage 

conclusions based on our audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions 

based on our audit objectives. 
Appendix II: Comments from the Department 

of Commerce 

Page 43 GAO-08-414 Improving and Measuring Census Coverage 

Appendix II: Comments from the Department 

of Commerce 

Note: GAO comments 

supplementing those in 

the report text appear at 

the end of this appendix. 
Appendix II: Comments from the Department 

of Commerce 

Page 44 GAO-08-414 Improving and Measuring Census Coverage 

See comment 4. 

See comment 3. 

See comment 2. 

See comment 1. 
Appendix II: Comments from the Department 

of Commerce 

Page 45 GAO-08-414 Improving and Measuring Census Coverage 

See comment 9. 

See comment 8. 

Now on p. 19. 

See comment 7. 

See comment 6. 

Now footnote 11, p. 16. 

See comment 5. 

Now footnote 4, p. 4. 
Appendix II: Comments from the Department 

of Commerce 

Page 46 GAO-08-414 Improving and Measuring Census Coverage 

See comment 14. 

See comment 13. 

See comment 12. 

See comment 11. 

See comment 10. 
Appendix II: Comments from the Department 

of Commerce 

Page 47 GAO-08-414 Improving and Measuring Census Coverage 

See comment 20. 

See comment 19. 

See comment 18. 

Now footnote 27, p. 32. 

See comment 17. 

Now footnote 26, p. 31. 

See comment 16. 

Now p. 29. 

See comment 15. 
Appendix II: Comments from the Department 

of Commerce 

Page 48 GAO-08-414 Improving and Measuring Census Coverage 

The following are our comments on the Department of Commerce’s letter 

dated April 3, 2008. 

We clarified our recommendation on relating components of coverage 

error to census operations in order to improve future decennials. 

Specifically, we recognize that the Bureau has developed table shells that 

associate components of coverage error to census operations, but these 

table shells and plans for how the Bureau will conduct its analyses should 

be shared with the Congress and other stakeholders. In addition, it is not 

clear from current Bureau plans how it will go about obtaining the data 

required to complete the tables. 

1. We have revised the Highlights page to reflect the Department of 

Commerce’s clarification that for the census coverage measurement 

program, the Bureau will expand its computer and clerical matching to 

match persons in its postenumeration survey. 

2. We have revised the report to reflect the Department of Commerce’s 

clarification that the Bureau will expand its computer and clerical 

matching to match persons in its postenumeration survey. 

3. In footnote 3, we have revised the report to clarify that the although 

the coverage measurement interview will start 4 months later 

compared to the Census 2000 telephone phase, person interviewing for 

the 2010 Census ends 1-1/2 months later than in 2000. 

4. In footnote 2, we have revised the sentence to indicate that the net 

error reflects the error remaining after omissions and erroneous 

enumerations offset one another. 

5. In footnote 4, we have revised the definition of the P-sample and Esample based on the Department of Commerce’s clarification. 

6. On page 16, we have clarified information about using the undercount 

probe during nonresponse follow-up and coverage follow-up. 

7. On page 18, we have clarified the information about the limited 

likelihood of duplication of individuals that will occur within a site of 

limited geographic size. 

8. On page 19, we have incorporated the clarifying information provided 

by the Department of Commerce. 

GAO Comments 
Appendix II: Comments from the Department 

of Commerce 

Page 49 GAO-08-414 Improving and Measuring Census Coverage 

9. On page 19, we have clarified information about the matching process. 

10. On page 20, we have revised the report to reflect that the 2008 Dress 

Rehearsal is intended to provide additional data to make final 

decisions about the Bureau’s implementation of coverage follow-up in 

2010. 

11. On page 22, we have revised the report to reflect that the Bureau will 

expand its computer and clerical matching to match persons in its 

postenumeration survey. 

12. On page 23, we have incorporated the Department of Commerce’s 

revisions on information that is collected during the census coverage 

measurement program. 

13. On page 23, we have deleted the incorrect sentence. 

14. On page 23, we have deleted the incorrect sentence. 

15. On page 26, we have revised the sentence to reflect that the Bureau 

will expand its computer and clerical matching to match persons in its 

postenumeration survey. 

16. On page 29, we revised the sentence to clarify the reason for refining 

the Bureau’s definition for erroneous enumeration. 

17. On page 31, we clarified the definition of the P-sample and E-sample as 

suggested. 

18. On page 31, for footnote 26, we acknowledged the change in clerical 

matching for the 2010 Census. 

19. On page 31, we deleted the incorrect sentence. 

20. On page 33, we provided attribution for the information to the National 

Academy of Sciences’ December 2007 interim report. 
Appendix III: GAO Contact and 

Staff Acknowledgments 

Page 50 GAO-08-414 Improving and Measuring Census Coverage 

Mathew J. Scirè (202) 512-6806 or sciremj@gao.gov 

In addition to the contact name above, Ron Fecso, Chief Statistician; Ernie 

Hazera, Assistant Director; Jenna Aurand; Betty Clark; and Jennifer 

Edwards made key contributions to this report. Tom Beall, Andrea Levine, 

Donna Miller, Lisa Mirel, and Elizabeth Wood provided significant 

technical support. 

Appendix III: GAO Contact and Staff 

Acknowledgments 

GAO Contact 

Acknowledgments 

(450557) 
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “E-mail Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, DC 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER