a

GAO

United States General Accounting Office

Report to the Chairman, Subcommittee 

on Civil Service, Census, and Agency 

Organization, Committee on Government 

Reform, House of Representatives

January 2003 2000 CENSUS

Coverage 

Measurement 

Programs’ Results, 

Costs, and Lessons 

Learned

GAO-03-287
As shown below, the two programs the Bureau employed to measure the 

quality of the 2000 Census population data did not meet their objectives. 

Coverage Measurement Programs Did Not Achieve Objectives

Program/objectives 

Objectives 

met? Reasons 

I.C.M. 

• Measure census coverage 

• Generate data for 

apportionment, redistricting, and 

federal programs using 

statistical sampling and 

estimation 

• Produce a “one-number” 

census 

No 

No 

No 

Program was canceled following January 

1999 U.S. Supreme Court ruling that the 

Census Act prohibits the use of sampling to 

apportion seats in the U.S. House of 

Representatives. 

A.C.E. 

• Measure census coverage 

• Generate data needed for 

redistricting and other purposes 

using statistical methods 

No 

No 

Uncertainties surrounding the accuracy of 

the A.C.E. results and the inability to 

resolve them in time to meet legally 

mandated deadlines for releasing data. 

Source: GAO. 

Note: This table reflects GAO’s analysis of U.S. Census Bureau data. 

The A.C.E. program achieved results other than those laid out in the 

Bureau’s formal objectives that highlight important lessons learned. They 

include (1) developing a coverage measurement methodology that is both 

operationally and technically feasible, (2) determining the level of geography 

at which coverage measurement is intended, (3) keeping stakeholders, 

particularly Congress, informed of the Bureau’s plans, and (4) adequately 

testing coverage measurement methodologies. It will be important for the 

Bureau to consider these as its current plans for the 2010 Census include 

coverage evaluation to measure the accuracy of the census but not 

necessarily to adjust the results. 

Of the roughly $207 million the Bureau obligated for I.C.M./A.C.E. programs 

from fiscal years 1996 through 2001, we identified about $22.3 million that 

was obligated for contracts involving over 170 vendors. We could not 

identify any obligations prior to 1996 in part because the Bureau included 

them with its general research and development efforts and did not assign 

the I.C.M./A.C.E. operations unique project codes in its financial 

management system. To track these costs in the future, it will be important 

for the Bureau to (1) have a financial management system that has specific 

project codes to capture coverage measurement costs, (2) establish the 

project codes as early in the planning process as possible, and (3) monitor 

the usage of the codes to ensure that they are properly charged. 

2000 CENSUS

Coverage Measurement Programs’ 

Results, Costs, and Lessons Learned 

www.gao.gov/cgi-bin/getrpt?GAO-03-287.

To view the full report, including the scope 

and methodology, click on the link above. 

For more information, contact Patricia A. 

Dalton at (202) 512-6806 or 

daltonp@gao.gov. 

Highlights of GAO-03-287, a report to 

Chairman, Subcommittee on Civil Service, 

Census, and Agency Organization, 

Committee on Government Reform, 

House of Representatives 

January 2003

To help measure the quality of the 

2000 Census and to possibly adjust 

for any errors, the U.S. Census 

Bureau (Bureau) conducted the 

Accuracy and Coverage Evaluation 

(A.C.E.) program. However, after 

obligating around $207 million for 

A.C.E. and its predecessor 

program, Integrated Coverage 

Measurement (I.C.M.), from fiscal 

years 1996 through 2001, the 

Bureau did not use either program 

to adjust the census numbers. 

Concerned about the amount of 

money the Bureau spent on I.C.M. 

and A.C.E. programs and what was 

produced in return, the 

subcommittee asked us to review 

the objectives and results of the 

programs, the costs of consultants, 

and how best to track future 

coverage measurement activities. 

The Secretary of Commerce should 

direct the Bureau to (1) work with 

Congress and other stakeholders 

and soon decide on whether and 

how coverage measurement will be 

used in 2010, (2) adopt lessons 

learned from its 2000 Census 

experience, and (3) ensure that its 

financial management systems can 

capture and report program 

activities early and that projects’ 

costs are monitored. The Bureau 

agreed with our recommendations 

but noted that for the 2000 Census, 

it followed the steps we identified 

as lessons learned. It also took 

exception to how we presented our 

conclusions concerning its ability 

to properly classify certain costs. 
Page i GAO-03-287 Census Coverage Measurement Programs 

Contents

Letter 1

Results in Brief 2

Background 4

Scope and Methodology 5

Coverage Measurement

Programs for the 2000 Census

Did Not Meet Bureau Objectives 7

I.C.M./A.C.E. Contractor Costs Are Not Complete 13

Tracking Future Coverage Measurement Costs 15

Conclusions 16

Recommendations for Executive Action 17

Agency Comments and Our Evaluation 17

Appendix

Appendix I: Comments from the Department of Commerce 20

Related GAO Products 23

Tables Table 1: Coverage Measurement Programs Did Not Achieve 

Objectives 7

Table 2: Census 2000 I.C.M./A.C.E. Contractor Costs for Fiscal 

Years 1996 through 2001 (Dollars in thousands) 14
Page 1 GAO-03-287 Census Coverage Measurement Programs 

United States General Accounting Office

Washington, D.C. 20548

A

January 29, 2003 Letr

The Honorable Dave Weldon, M.D.

Chairman

Subcommittee on Civil Service,

Census, and Agency Organization

Committee on Government Reform

House of Representatives

Dear Mr. Chairman:

To assess the quality of population data for the 2000 Census and to possibly 

adjust for any errors, the U.S. Census Bureau (Bureau) conducted the 

Accuracy and Coverage Evaluation (A.C.E.) program. The A.C.E. program 

was first included in Bureau program documents in November 1998 and 

funded for fiscal years 2000 through 2002 with proposed funding through 

December 31, 2002. Its predecessor, Integrated Coverage Measurement 

(I.C.M.), began in May 1995 and was funded by the Bureau for fiscal years 

1996 through 1999. The Bureau obligated about $207 million to both 

programs from fiscal years 1996 through 2001,1

 which was about 3 percent 

of the $6.5 billion total estimated cost of the 2000 Census. However, neither 

program was used to adjust the 2000 Census population count. 

Concerned about the amount of money the Bureau spent on both programs 

as well as what was received in return for its investment, you and former 

Vice Chairman Dan Miller asked us to examine (1) the Bureau’s objectives 

for the I.C.M./A.C.E. programs and the extent to which those objectives 

were met, (2) the cost of consultants and technical studies, and (3) ways to 

track the costs of coverage measurement activities in future censuses. This 

report responds to that request.

In October 2002, we issued a report that provides additional information on 

the cost of the I.C.M./A.C.E. programs.2

 Both reports are part of our 

ongoing series on the results of the 2000 Census and the lessons learned for 

planning a more cost-effective census in 2010. (See the Related GAO 

1

At the time of our report, obligated costs after fiscal year 2001 were not final.

2

U.S. General Accounting Office, 2000 Census: Complete Costs of Coverage Evaluation 

Programs Are Not Available, GAO-03-41 (Washington, D.C.: Oct. 31, 2002).
Page 2 GAO-03-287 Census Coverage Measurement Programs 

Products section at the end of this report for the assessments issued to 

date.)

Results in Brief The coverage measurement programs the Census Bureau planned for the 

2000 Census—I.C.M. and its successor, A.C.E.—did not meet their intended 

objectives. Although the Bureau designed I.C.M. to measure census 

coverage; generate data for apportionment, redistricting, and federal 

programs using statistical sampling and estimation; and produce a “onenumber” census based on statistical sampling and estimation, the Bureau 

abandoned the program following a Supreme Court ruling that the Census 

Act prohibited the use of statistical sampling to generate population data 

for reapportioning the House of Representatives.3

 Because its 

replacement—A.C.E.—did not provide a reliable measure of census 

accuracy in time to meet legally mandated deadlines for releasing 

redistricting data, the Bureau decided against using it to adjust the census 

data for nonapportionment purposes. 

The difficulties the Bureau encountered in trying to implement I.C.M. and 

A.C.E. underscore the importance of (1) developing a coverage 

measurement methodology that is both technically and operationally 

feasible, (2) determining the level of geography at which coverage 

measurement is intended, (3) keeping stakeholders, particularly Congress, 

informed of the Bureau’s plans, and (4) adequate testing. It will be 

important for the Bureau to address these lessons learned as its planning 

efforts for the 2010 Census continue. Those plans currently call for a 

coverage measurement program to evaluate the accuracy of the census, 

whereas the issue of whether coverage measurement will be used to adjust 

the numbers has not yet been resolved.

Concerning the cost of contracts, including the consultants and technical 

studies related to the I.C.M./A.C.E. programs, we identified about 

$22.3 million of obligated amounts for over 170 vendors for fiscal year 1996 

through fiscal year 2001 from unaudited Bureau financial management 

reports. This amount does not represent the complete contract costs for 

the I.C.M./A.C.E. programs for three reasons. First, the Bureau considered 

costs from earlier years to be part of its general research and development 

efforts and did not assign unique project codes to identify I.C.M./A.C.E. 

3

Department of Commerce v. United States House of Representatives, 525 U.S. 316 (1999).
Page 3 GAO-03-287 Census Coverage Measurement Programs 

programs and related costs in its financial management system. Second, 

although $182,000 of fiscal year 1996 obligated costs were identified in the 

Bureau’s financial management systems as contract costs for an I.C.M. 

special test, the Bureau did not consider these costs as part of the 

I.C.M./A.C.E. programs and classified these costs as general research. We 

disagreed with the Bureau on this point and have included this amount in 

our report as part of the I.C.M./A.C.E. contract costs that we could identify 

from Bureau records. Finally, certain costs, such as program evaluations, 

were Bureau-wide in nature, and the portion attributable to I.C.M./A.C.E 

could not be separated out. 

The Bureau’s ability to track future costs of coverage measurement 

activities is primarily dependent on its ability to (1) ensure that its financial 

management system accurately and completely captures the accounting 

and reporting of project codes, (2) design its project codes to capture 

coverage measurement activities as early in the planning process as 

possible, even though the activities’ names may change as the programs 

evolve, and (3) correctly charge the project codes established. 

Although the Bureau has never used the results of its coverage 

measurement programs to adjust census numbers, we believe that an 

evaluation of the accuracy of the census is essential given the importance 

of the data, the need to know the nature of any errors, and the cost of the 

census overall. Whether the results of the evaluation should be used to 

adjust the census is still an open question, the answer to which should 

involve discussions between the Bureau, Congress, and other stakeholders, 

and be based on detailed data and a convincing demonstration of the 

feasibility of the Bureau's proposed approach. Regardless of the outcome 

of the decision, it is critical that it be made soon so that the Bureau can 

proceed with its planning. The longer the Bureau goes without a firm 

decision on the role of coverage measurement, the greater the risk of 

wasted resources and disappointing results. In light of the challenges 

facing the Bureau as it prepares for the next decennial census in 2010, we 

recommend that the Secretary of Commerce direct the U.S. Census Bureau 

to 

• in conjunction with Congress and other stakeholders, decide soon on 

whether and how coverage measurement will be used in the 2010 

Census;

• consider incorporating lessons learned from its coverage measurement 

experience during the 2000 Census, such as (1) demonstrating both the 
Page 4 GAO-03-287 Census Coverage Measurement Programs 

operational and technical feasibility of its coverage measurement 

methods, (2) determining the level of geography at which coverage can 

be reliably measured, (3) keeping Congress and other stakeholders 

informed of its plans, and (4) adequately testing coverage measurement 

prior to full implementation; and 

• ensure that the Bureau’s financial management systems can capture and 

report program activities early in the decennial process and that project 

costs are monitored for accuracy and completeness. 

The Secretary of Commerce forwarded written comments from the Bureau 

of the Census on a draft of this report (see app. I). The Bureau agreed with 

our recommendations noting that they were important steps that should be 

followed in the development of a coverage measurement methodology for 

the 2010 Census. However, the Bureau maintained that most of these steps 

such as keeping Congress and other stakeholders informed of its plans 

were followed for the 2000 Census. We disagree because, as noted in the 

report, the lack of information contributed to stakeholders’ skepticism 

surrounding the Bureau’s plans. The Bureau also took exception to the way 

we presented our conclusions concerning its ability to properly classify 

certain costs associated with the development of the Bureau’s coverage 

measurement programs. Our perspective on the Bureau’s position is 

detailed in the “Agency Comments and Our Evaluation” section at the end 

of this report. 

Background The Bureau puts forth tremendous effort to conduct a complete and 

accurate count of the nation’s population. However, some degree of error in 

the form of persons missed or counted more than once is inevitable 

because of limitations in census-taking methods. Because census results 

are used, among other purposes, to apportion Congress, redraw 

congressional districts, and allocate federal aid to state and local 

governments, the size and demographic composition of these coverage 

errors have become increasingly sensitive since the Bureau was first able 

to generate detailed data on them during the 1980 Census. However, the 

Bureau has never used the results of its coverage measurements to correct 

estimated coverage errors.

The Bureau first attempted to measure the accuracy of the census in the 

1940s when it compared the census numbers to birth and death certificates 

and other administrative data using a procedure called demographic 

analysis. Modern coverage measurement began with the 1980 Census when 
Page 5 GAO-03-287 Census Coverage Measurement Programs 

the Bureau compared census figures to the results of an independent 

sample survey of the population. Using statistical methods, the Bureau 

generated detailed measures of the differences among undercounts of 

particular ethnic, racial, and other groups. In the months that followed, 

many lawsuits were filed, most contending that the results of the 1980 

coverage measurement should have been used to adjust the census. 

However, the Bureau designed the evaluation to measure errors, not to 

correct the census results, and the Director of the Census Bureau decided 

against adopting the adjusted numbers, as they were deemed flawed due to 

missing and inaccurate data.

The quality of the coverage measurement data improved for the 1990 

Census, and the Bureau recommended statistically adjusting the results. 

However, the Secretary of Commerce determined that the evidence to 

support an adjustment was inconclusive and decided not to adjust the 1990 

Census. The adjustment decision was complicated by the fact that the 1990 

Census figures had already been released when the coverage measurement 

results became available in the spring of 1991. The Secretary of Commerce 

was concerned that two sets of numbers—the actual census results and the 

adjusted figures—could create confusion and might allow political 

considerations to play a part in choosing between sets of numbers when 

the outcome of the choices, such as congressional apportionment, could be 

known in advance of a decision.

Scope and 

Methodology

To determine the objectives of 2000 Census I.C.M./A.C.E. programs and 

their results, we reviewed Bureau and other documents that included 

Federal Register notices; Census Operational Plans; reports to Congress; 

internal memorandums; research and feasibility studies; and reports of the 

Executive Steering Committee for Accuracy and Coverage Policy (ESCAP) 

I and II, which assessed the results of the A.C.E. program and 

recommended how they should be used.

To determine costs for consultants and technical studies for 2000 Census 

I.C.M./A.C.E. programs, we focused on object class code 25 from the 

financial management reports to obtain contract data. With Bureau 

assistance, we identified I.C.M./A.C.E. project accounts and analyzed 

amounts by fiscal year using the financial management reports generated 

by the Department of Commerce’s Administrative Management System 

(CAMS). We reviewed and analyzed obligated and expended data for all 

coverage measurement programs that existed during the 2000 Census for 
Page 6 GAO-03-287 Census Coverage Measurement Programs 

fiscal years 1991 to 2003. We did not audit financial data provided by the 

Bureau. 

To determine ways to track future costs, we reviewed current Bureau 

financial management reports and considered established standards of 

accounting, auditing, and internal controls.

In addition, we met with key Bureau officials to discuss the results of our 

analysis and obtain their observations and perspectives. The limitations we 

encountered in the scope of our work on this assignment are as follows.

• We were unable to determine the complete contractual and technical 

studies costs of the I.C.M./A.C.E. programs because the Bureau 

considered any I.C.M./A.C.E.-related costs from fiscal years 1991 

through 1995 as part of its general research and development programs 

and thus did not separately track these costs. Although some costs were 

tracked in fiscal year 1996, the Bureau still considered these costs as 

research and development and did not include these costs as 

I.C.M./A.C.E. program costs.

• We were unable to identify I.C.M./A.C.E. portions of costs from projects 

that covered the entire census, such as the 2000 Census Evaluation 

program. 

• We did not evaluate the propriety of contracts for I.C.M./A.C.E. 

programs.

Our work was performed in Washington, D.C., and at U.S. Census Bureau 

headquarters in Suitland, Maryland, from June 2002 through October 2002 

in accordance with generally accepted government auditing standards. On 

January 7, 2003, the Secretary of Commerce provided written comments on 

a draft of this report. We address these comments in the “Agency 

Comments and Our Evaluation” section, and have reprinted them in 

appendix I.
Page 7 GAO-03-287 Census Coverage Measurement Programs 

Coverage 

Measurement

Programs for the 2000 

Census

Did Not Meet Bureau 

Objectives 

In planning the 2000 Census, the Bureau developed a new coverage 

measurement program, I.C.M., that was designed to address the major 

shortcomings of the 1990 coverage measurement program. However, as 

shown in table 1, much like similar programs in earlier censuses, the 

Bureau did not use I.C.M. and its successor program, A.C.E., to adjust the 

census because of legal challenges, technical obstacles, and the inability to 

resolve uncertainties in the data in time to meet the deadlines for releasing 

the data.

Table 1: Coverage Measurement Programs Did Not Achieve Objectives

Source: GAO.

Note: This table reflects GAO’s analysis of U.S. Census Bureau data.

The Bureau Canceled

I.C.M. in Response to a

Supreme Court Ruling

In designing I.C.M., the Bureau’s goal was to produce a single, consolidated 

count or “one-number” census and thus avoid the controversy of having 

two sets of census results as occurred during the 1990 Census. Thus, as 

shown in table 1, the objectives of I.C.M. were to (1) measure census 

coverage, (2) generate, using statistical sampling and estimation methods, 

the detailed data required for apportionment, congressional redistricting, 

and federal program purposes, and (3) produce a one-number census.

Program/objectives

Objectives 

met? Reasons

I.C.M.

• Measure census coverage.

• Generate data for apportionment, 

redistricting, and federal program 

purposes using statistical 

sampling and estimation.

• Produce a “one-number” census.

No.

No.

No.

Program was canceled following 

January 1999 U.S. Supreme Court 

ruling that the Census Act prohibits the 

use of sampling to produce counts 

used to apportion seats in the U.S. 

House of Representatives. The Bureau 

then replaced I.C.M. with the A.C.E. 

program.

A.C.E.

• Measure census coverage.

• Generate data needed for 

redistricting and federal program 

purposes statistical methods.

No.

No.

A.C.E. results were not used because 

of uncertainties surrounding the 

accuracy of the results and the inability 

to resolve them in time to meet legally 

mandated deadlines for releasing data. 
Page 8 GAO-03-287 Census Coverage Measurement Programs 

The Bureau’s plans for I.C.M. emerged in response to the unsatisfactory 

results of the 1990 Census. Although the 1990 headcount was, at that time, 

the most costly in U.S. history, it produced data that were less accurate 

than those from the 1980 Census. The disappointing outcome was due in 

large part to the Bureau’s efforts to count housing units that did not mail 

back their census questionnaires. The operation, known as nonresponse 

follow-up, where enumerators visited and collected information from each 

nonresponding housing unit, proved to be costly and error-prone when a 

higher-than-expected workload and a shortage of enumerators caused the 

operation to fall behind schedule. The final stages of nonresponse followup were particularly problematic. Indeed, while enumerators finished 90 

percent of the follow-up workload within 8 weeks (2 weeks behind 

schedule), it took another 6 weeks to resolve the remaining 10 percent. 

Moreover, in trying to complete the last portion of nonresponse follow-up 

cases, the Bureau accepted less complete responses and information from 

nonhousehold members such as neighbors, which may have reduced the 

quality of the data.

In the years following the 1990 Census, Congress, the Bureau, several 

organizations, and GAO, concluded that fundamental design changes were 

needed to reduce census costs and improve the quality of the data. In 

response, the Bureau reengineered a number of operations for the 2000 

Census.

For example, to save time and reduce its nonresponse follow-up workload, 

the Bureau planned to enumerate a sample of the last remaining portion of 

nonresponse follow-up cases instead of visiting every nonresponding 

household as it had done in previous censuses. To adjust for enumeration 

errors, the Bureau developed I.C.M., which was intended to reconcile the 

original census figures with data obtained from a separate, independent 

count of a sample of 750,000 housing units using a statistical process called 

Dual System Estimation. The Bureau believed that this approach offered 

the best combination of reduced costs, improved accuracy expected at 

various geographic levels, and operational feasibility. 
Page 9 GAO-03-287 Census Coverage Measurement Programs 

However, concerned about the legality of the Bureau’s planned use of 

sampling and estimation, members of Congress challenged the Bureau’s 

use of I.C.M. in court. In January 1999, the Supreme Court ruled that the 

Census Act4

 prohibited the use of statistical sampling to generate 

population data for reapportioning the House of Representatives.5

A.C.E. Did Not Meet Bureau 

Objectives

Following the Supreme Court ruling, the Bureau planned to produce 

apportionment numbers using traditional census-taking methods, and 

provide statistically adjusted numbers for nonapportionment uses of the 

data such as congressional redistricting and allocating federal funds. The 

Bureau initiated the A.C.E. program, which was designed to take a national 

sample of approximately 300,000 housing units to evaluate coverage errors 

among different population groups and statistically correct for them. Thus, 

as shown in table 1, the Bureau’s objectives for A.C.E. were to (1) measure 

how many people were missed in the census and how many were 

erroneously included and (2) produce the detailed data required in time for 

redistricting and federal program purposes. 

However, while the Bureau generally conducted A.C.E. in accordance with 

its plans,6

 the Bureau later determined that the A.C.E. results did not 

provide a reliable measure of census accuracy and could not be used to 

adjust the nonapportionment census data. 

4

13 U.S.C. 195.

5

Department of Commerce v. United States House of Representatives 525, U.S. 316 (1999).

6

See, for example, U.S. General Accounting Office, 2000 Census: Coverage Evaluation 

Interviewing Overcame Challenges, but Further Research Needed, GAO-02-26 (Washington, 

D.C.: Dec. 31, 2001), and 2000 Census: Coverage Evaluation Matching Implemented as 

Planned, but Census Bureau Should Evaluate Lessons Learned, GAO-02-297 (Washington, 

D.C.: Mar. 14, 2002).
Page 10 GAO-03-287 Census Coverage Measurement Programs 

The first decision against A.C.E. occurred in March 2001, when the Acting 

Director of the Census Bureau recommended to the Secretary of 

Commerce that the unadjusted census data be used for redistricting 

purposes. He cited as a primary reason an apparent inconsistency between 

the population growth over the prior decade, as implied by A.C.E. results, 

and demographic analysis, which estimated the population using birth, 

death, and other administrative records. The inconsistency raised the 

possibility of an unidentified error in either the A.C.E. or census numbers. 

He reported that the inconsistency could not be resolved prior to April 1, 

2001, the legally mandated deadline for releasing redistricting data.7

The second decision against A.C.E. came in October 2001 when, based on a 

large body of additional research, ESCAP decided against adjusting census 

data for allocating federal aid and other purposes, because A.C.E. failed to 

identify a significant number of people erroneously included in the census, 

and other remaining uncertainties. According to Bureau officials, it might 

be possible to use adjusted data to produce intercensal population 

estimates for federal programs that require this information; however, the 

Bureau would need to revise the A.C.E. results before any use of the data 

could be considered. 

The Bureau’s Experience in 

Implementing

Coverage Measurement 

Programs 

Highlights Important 

Lessons Learned

Although I.C.M. and A.C.E. did not meet their formal objectives, they did 

produce a body of important lessons learned. As the Bureau’s current 

approach for the 2010 Census includes coverage measurement to assess 

the accuracy of the census (but not necessarily to adjust the numbers 

themselves), it will be important for the Bureau to consider these lessons 

as its planning efforts continue. The lessons include (1) developing a 

coverage measurement methodology that is both technically and 

operationally feasible, (2) determining the level of geography at which 

coverage measurement is intended, (3) keeping stakeholders, particularly 

Congress, informed of the Bureau’s plans, and (4) adequately testing the 

eventual coverage measurement program.

1. A.C.E. demonstrated operational, but not technical feasibility. 

According to Bureau officials, an important result of the A.C.E. 

program was that it demonstrated, from an operational perspective 

only, the feasibility of conducting a large independent field check on 

7

March 1, 2001, Memorandum to Secretary Donald Evans from Acting Director William 

Barron, Jr. (ESCAP I decision memo.)
Page 11 GAO-03-287 Census Coverage Measurement Programs 

the quality of the census. The Bureau canvassed the entire A.C.E. 

sample area to develop an address list, collected census response data 

for persons living in the sample areas on census day, and conducted an 

operation to try and match A.C.E. respondents to census respondents, 

all independent of the regular census operations and within required 

time frames.

Our separate reviews of two of these operations—interviewing 

respondents and matching A.C.E. and census data—while raising 

questions about the impact on final A.C.E. results due to apparently 

small operational deviations, also concluded that the Bureau 

implemented those two operations largely as planned.8

Nevertheless, while the Bureau demonstrated that it could execute 

A.C.E. field operations using available resources within required time 

frames, as the Bureau has noted, feasibility also consists of a technical 

component—that is, whether the A.C.E. methodology would improve 

the accuracy of the census. Although the Bureau clearly stated in its 

justification for A.C.E. that the effort would make the census more 

accurate, as noted earlier, because of unresolved data discrepancies, its 

experience in 2000 proved otherwise. Moreover, according to the 

Bureau, because the A.C.E. was designed to correct a census with a net 

coverage error similar to that observed in previous censuses, the 

Bureau commented that applying the methodology to the historically 

low levels of net error observed in the 2000 Census represented a 

unique and unanticipated challenge for A.C.E. Thus, it will be important 

for the Bureau to refine its coverage measurement methodology to 

ensure that it is technically feasible. 

8

See GAO-02-26 and GAO-02-297.
Page 12 GAO-03-287 Census Coverage Measurement Programs 

2. The level of geography at which the Bureau can successfully measure 

coverage is unclear. Since the October 2001 decision to not rely on 

adjusted census data for nonapportionment and nonredistricting 

purposes, Bureau officials have told us that they now doubt whether 

census data can reliably be improved down to the level of geography 

for which A.C.E. was intended to improve the accuracy—the census 

tract level (neighborhoods that typically contain around 1,700 housing 

units and 4,000 people). The Bureau’s current position differs from that 

taken in 2000, when it reported to Congress that it expected accuracy at 

the tract level to be improved, on average, by A.C.E. statistically 

adjusting numbers at an even lower level of geography—the census 

block level.9

 Uncertainty in the level of geography at which accuracy is 

to be measured or improved can affect the overall design of coverage 

measurement, as well as its technical feasibility. Therefore, it will be 

important for the Bureau to determine the level of geography at which 

it intends to measure accuracy as it decides the role and design of 

future coverage measurement programs.

3. Keeping stakeholders informed is essential. Throughout the 1990s, 

Congress and other stakeholders, including GAO, expressed concerns 

about the Bureau’s planned use of sampling and statistical estimation 

procedures to adjust the census. A key cause of this skepticism was the 

Bureau’s failure to provide sufficiently detailed data on the effects that 

I.C.M. would have at different levels of geographic detail. Information 

was also lacking on the various design alternatives being considered, 

their likely implications, and the basis for certain decisions. As a result, 

it was difficult for Congress and other stakeholders to support the 

Bureau’s coverage measurement initiatives.

9

U.S. Census Bureau, “Report to Congress—The Plan for Census 2000,” 44-46 and Accuracy and 

Coverage Evaluation—Statement on the Feasibility of Using Statistical Methods to Improve 

the Accuracy of the Census 2000 (June 2000), 19, fn. 19.
Page 13 GAO-03-287 Census Coverage Measurement Programs 

For example, on September 24, 1996, the House Committee on 

Government Reform and Oversight issued a report that criticized the 

Bureau’s initiatives for sampling and statistical estimation. Among 

other things, the Committee found that the Bureau had not clarified 

issues of accuracy, particularly for small geographic areas, raised by the 

sampling initiative. Congress’s perspective on the process was later 

reflected in its enactment of legislation in 1997 that included provisions 

requiring the Department of Commerce to provide Congress with 

comprehensive information on its planned use of statistical estimation 

within 30 days.10

4. Adequate testing of coverage measurement methodologies is critical. 

Although the Bureau conducted a dress rehearsal for the census in 

three locations across the country that was intended to demonstrate 

the overall design of the census, the 1998 operation did not reveal the 

problems that the Bureau encountered in dealing with the 

discrepancies between the 2000 A.C.E. results and its benchmarks. 

According to Bureau officials, this was partly because the sites were 

not representative of the nation at large. Additionally, as a result of a 

compromise between Congress and the administration to 

simultaneously prepare for a nonsampling census, the I.C.M. was tested 

at only two of the three dress rehearsal sites—an urban area and an 

Indian reservation—but was not tested in a rural location as was 

originally planned. An earlier test in 1995 was also not comprehensive 

in that it did not test a sampling operation designed to help determine 

whether nonresponse follow-up of the magnitude projected by the 

Bureau’s current plan could be completed in time for the I.C.M to be 

done on schedule. 

I.C.M./A.C.E. 

Contractor Costs Are 

Not Complete

From fiscal year 1996 through fiscal year 2001, the Bureau obligated about 

$207 million for I.C.M./A.C.E. activities.11 As shown in table 2, of that 

$207 million, we identified about $22.3 million (11 percent) in obligated 

amounts for contracts involving more than 170 vendors. These contracts 

were primarily for technical advisory and assistant services, computer 

systems support, and training. 

10Pub. L. 105-18, Title VIII, June 12, 1997.

11At the time of our report, obligated costs after fiscal year 2001 were not final.
Page 14 GAO-03-287 Census Coverage Measurement Programs 

Table 2: Census 2000 I.C.M./A.C.E. Contractor Costs for Fiscal Years 1996 through 2001 (Dollars in thousands)

Source: GAO.

Note: This table reflects GAO’s analysis of U.S. Census Bureau financial management reports.

Although the Bureau tracked some costs of contracts for the I.C.M./A.C.E. 

programs, we found that the $22.3 million did not represent the complete 

contractor costs of the programs because of the following three factors.

• First, the Bureau only tracked the contractor costs associated with 

conducting the I.C.M./A.C.E. programs, which covers the period from 

fiscal year 1997 through 2003. Although life cycle costs for the 2000 

Census cover a 13-year period from fiscal years 1991 through 2003, 

senior Bureau officials said that the I.C.M./A.C.E. program was not 

viable for implementation until fiscal year 1997. Therefore, the Bureau 

considered contractor costs from earlier years as part of its general 

research and development programs, and the Bureau did not assign 

unique project codes to identify I.C.M./A.C.E. programs and related 

costs in its financial management system.

• Second, although $182,000 of fiscal year 1996 obligated contractor costs 

were identifiable in the Bureau’s financial management system as an 

I.C.M. special test, the Bureau did not consider these costs as part of the 

I.C.M./A.C.E. programs. Instead, these costs were considered general 

research and development. However, because the Bureau separately 

identified these costs as I.C.M. program contractor costs, we have 

Project description FY96 FY97 FY98 FY99 FY00 FY01 Total

A.C.E. coverage management 0 0 0 0 $1,458 0 $1,458

A.C.E. operations 0 0 0 0 249 ($2) 247

I.C.M. collection 0 0 $201 $41 6,272 467 6,981

Framework 3 total 0 0 201 41 7,979 465 8,686

I.C.M. procedures and training 0 $249 594 1,065 1,766 814 4,488

I.C.M. processing 0 0 0 2,412 287 574 3,273

Framework 5 total 0 249 594 3,477 2,053 1,388 7,761

I.C.M. dress rehearsal 0 0 502 20 0 (10) 512

I.C.M. special test $182 628 0 0 0 0 810

I.C.M. coverage measurement 0 409 1,257 1,991 (130) 999 4,526

Framework 6 total 182 1,037 1,759 2,011 (130) 989 5,848

Total $182 $1,286 $2,554 $5,529 $9,902 $2,842 $22,295
Page 15 GAO-03-287 Census Coverage Measurement Programs 

included the $182,000 as part of the I.C.M./A.C.E. program contractor 

costs in this report.

• Finally, we were unable to identify the I.C.M./A.C.E. portions of costs 

that were part of other programs. For example, in late fiscal year 2000 

and after, the Bureau did not separate A.C.E. evaluations from its other 

2000 Census evaluations in its financial management systems. Bureau 

officials stated that the contracts for evaluations included overall 2000 

Census and A.C.E. evaluations, and did not have a separate code 

identifying A.C.E. costs. 

Tracking Future 

Coverage 

Measurement Costs

During the 2000 Census, the Bureau, its auditors, and GAO, found extensive 

weaknesses in the Bureau’s financial management system, the components 

of which include hardware, software, and associated personnel. The 

weaknesses included difficulties in providing reliable and timely financial 

information to manage current government operations and preparing 

financial statements and other reports. Together, they affected the 

completeness, accuracy, and timeliness of data needed for informed 

management decisions and effective oversight. In light of these 

weaknesses, the Bureau’s ability to track future costs of coverage 

measurement activities will largely depend on three factors. 

• First, a sound financial management system is critical. As discussed in 

our December 2001 report, the Bureau’s core financial management 

system, CAMS, had persistent internal control weaknesses in fiscal year 

2000.12 In its latest financial report, the Bureau indicated that these 

weaknesses have continued through fiscal year 2001.13 The Bureau 

expects to issue its fiscal year 2002 financial report shortly.

• Second, it would be important to set up project codes to capture 

coverage measurement activities as early in the planning process as 

possible. The Bureau did not set up a specific project code to identify 

I.C.M. program costs until 1996 because, according to Bureau officials, 

12U.S. General Accounting Office, 2000 Census: Analysis of Fiscal Year 2000 Budget and 

Internal Control Weaknesses at the U.S. Census Bureau, GAO-02-30 (Washington, D.C.: 

Dec. 28, 2001).

13U.S. Census Bureau, 2001 Financial Report (Suitland, Md.: May 2002).
Page 16 GAO-03-287 Census Coverage Measurement Programs 

the I.C.M. program was not viable until 1997 and all costs up to this 

point were considered general research. 

• Finally, it would be important for Bureau personnel to correctly charge 

the project codes established for the coverage measurement program 

activities. During the 2000 Census, for example, while the Bureau 

established a project code and a budget for the remote Alaska 

enumeration, the project costs were erroneously charged to and 

commingled with a project code for enumerating special populations. As 

a result, the actual costs for remote Alaska enumeration were reported 

by the Bureau’s financial management system as zero and are unknown, 

while enumerating special population costs are overstated. 

Conclusions The Bureau’s 2000 Census coverage measurement programs did not 

achieve their primary objectives of measuring the accuracy of the census 

and adjusting the results because of legal challenges, technical hurdles, and 

questionable data. However, beyond these formal objectives, there 

emerged several important lessons learned that Bureau managers should 

consider because current plans for the 2010 Census include coverage 

measurement. At the same time, it will also be important for the Bureau to 

be capable of fully tracking the money it spends on coverage measurement 

and other census activities so that Congress and other stakeholders can 

hold the Bureau accountable for achieving intended results.

Although the Bureau has never used the results of its coverage 

measurement programs to adjust census numbers, we believe that an 

evaluation of the accuracy and completeness of the census is critical given 

the many uses of census data, the importance of identifying the magnitude 

and characteristics of any under- and overcounts, and the cost of the 

census overall. Less clear is whether the results of the coverage 

measurement should be used to adjust the census. Any Bureau decisions 

on this matter should involve close consultation with Congress and other 

stakeholders, and be based on detailed data and a convincing 

demonstration of the feasibility of the Bureau's proposed approach. 

Whatever the decision, it is imperative that it be made soon so that the 

Bureau can design appropriate procedures and concentrate on the business 

of counting the nation’s population. The longer the 2010 planning process 

proceeds without a firm decision on the role of coverage measurement, the 

greater the risk of wasted resources and disappointing results.
Page 17 GAO-03-287 Census Coverage Measurement Programs 

Recommendations for 

Executive Action

To help ensure that any future coverage measurement efforts achieve their 

intended objectives and costs can be properly tracked, we recommend that 

the Secretary of Commerce direct the Bureau to 

• in conjunction with Congress and other stakeholders, come to a 

decision soon on whether and how coverage measurement will be used 

in the 2010 Census;

• consider incorporating lessons learned from its coverage measurement 

experience during the 2000 Census, such as (1) demonstrating both the 

operational and technical feasibility of its coverage measurement 

methods, (2) determining the level of geography at which coverage can 

be reliably measured, (3) keeping Congress and other stakeholders 

informed of its plans, and (4) adequately testing coverage measurement 

prior to full implementation; and

• ensure that the Bureau’s financial management systems can capture and 

report program activities early in the decennial process and ensure that 

project costs are monitored for accuracy and completeness. 

Agency Comments and 

Our Evaluation

The Secretary of Commerce forwarded written comments from the Census 

Bureau on a draft of this report, which are reprinted in appendix I. The 

Bureau agreed with our recommendations highlighting the steps that 

should be followed in the development of a coverage measurement 

methodology for the 2010 Census and acknowledged their importance. 

However, the Bureau maintained that it followed most of these steps for 

the 2000 Census including (1) keeping stakeholders, particularly Congress, 

informed of the Bureau’s plans, (2) determining the level of geography at 

which coverage measurement is intended, and (3) adequately testing 

coverage measurement methodologies. The Bureau also maintained that 

throughout the 1990s, it had an open and transparent process for 

implementing the coverage measurement program, including the levels of 

geography to which its results would be applied. 
Page 18 GAO-03-287 Census Coverage Measurement Programs 

We disagree. As we stated in our report, the Bureau’s failure to provide 

important information was a key cause of congressional skepticism over 

the Bureau’s coverage measurement plans. In fact, Congress was so 

concerned about the lack of comprehensive information on the Bureau’s 

proposed approach that in July 1997, it passed a law that included 

provisions requiring the Department of Commerce to provide detailed data 

on the Bureau’s planned use of statistical estimation within 30 days.14 We 

revised the report to include this, and provide other examples to further 

support our position that the Bureau’s I.C.M. and A.C.E. planning and 

development processes were less than fully open and transparent. 

The Bureau also commented that each major component of the 

I.C.M./A.C.E. program underwent “rigorous” testing in the middle of the 

decade as well as during the dress rehearsal for the 2000 Census held in 

1998. We believe this overstates what actually occurred. As we noted in the 

report, the dress rehearsal failed to detect the problems that A.C.E. 

encountered during the 2000 Census because the sites were not 

representative of the nation. Additionally, because of an agreement 

between Congress and the administration to simultaneously prepare for a 

census that did not include sampling, the I.C.M. was only tested at two of 

the three dress rehearsal sites—an urban area and an Indian reservation—

but was not tested in a rural location as was originally planned. We made 

this and other revisions to strengthen our point. 

Because the A.C.E. was designed to correct a census with a net coverage 

error similar to that observed in previous censuses, the Bureau commented 

that applying the methodology to the historically low levels of net error 

observed in the 2000 Census represented a unique and unexpected 

challenge for A.C.E. We revised the report to reflect this additional context. 

The Bureau took exception to the way we presented our conclusions 

concerning its ability to properly classify certain costs associated with the 

development of the Bureau’s coverage measurement programs. The Bureau 

noted that it decided not to separately track coverage measurement 

development costs in 1994, because there was no internal or external 

request for a separate cost accounting of the program.

Our report does not make interpretive conclusions or qualitative judgments 

about which coverage measurement program costs the Bureau decided to 

14Pub. L. 105-18, Title VIII, June 12, 1997.
Page 19 GAO-03-287 Census Coverage Measurement Programs 

track. Instead, the report (1) points out that we could not identify all of the 

contractor costs associated with the I.C.M./A.C.E. programs because of the 

three factors described in the report, and (2) underscores the importance 

of a sound financial management system for tracking, planning, and 

development costs for the 2010 Census. 

We are sending copies of this report to other interested congressional 

committees, the Secretary of Commerce, and the Director of the U.S. 

Census Bureau. Copies will be made available to others upon request. This 

report will also be available at no charge on GAO’s home page at 

http://www.gao.gov.

Please contact Patricia A. Dalton on (202) 512-6806 or by E-mail at 

daltonp@gao.gov if you have any questions. Other key contributors to this 

report were Robert Goldenkoff, Roger Stoltz, Carolyn Samuels, Cindy 

Brown-Barnes, Ty Mitchell, and Linda Brigham.

Sincerely yours,

Patricia A. Dalton

Director

Strategic Issues

McCoy Williams

Director

Financial Management and Assurance
Page 20 GAO-03-287 Census Coverage Measurement Programs 

Appendix I

Comments from the Department of Apendxies

Commerce ApendxiI
Appendix I

Comments from the Department of 

Commerce

Page 21 GAO-03-287 Census Coverage Measurement Programs 
Appendix I

Comments from the Department of 

Commerce

Page 22 GAO-03-287 Census Coverage Measurement Programs 
Page 23 GAO-03-287 Census Coverage Measurement Programs 

Related GAO Products

2000 Census: Complete Costs of Coverage Evaluation Programs Are Not 

Available. GAO-03-41. Washington, D.C.: October 31, 2002.

2000 Census: Lessons Learned for Planning a More Cost-Effective 2010 

Census. GAO-03-40. Washington, D.C.: October 31, 2002.

2000 Census: Refinements to Full Count Review Program Could Improve 

Future Data Quality. GAO-02-562. Washington, D.C.: July 3, 2002.

2000 Census: Coverage Evaluation Matching Implemented as Planned, 

but Census Bureau Should Evaluate Lessons Learned. GAO-02-297. 

Washington, D.C.: March 14, 2002.

2000 Census: Best Practices and Lessons Learned for More Cost-Effective 

Nonresponse Follow-up. GAO-02-196. Washington, D.C.: February 11, 2002.

2000 Census: Coverage Evaluation Interviewing Overcame Challenges, 

but Further Research Needed. GAO-02-26. Washington, D.C.: 

December 31, 2001.

2000 Census: Analysis of Fiscal Year 2000 Budget and Internal Control 

Weaknesses at the U.S. Census Bureau. GAO-02-30. Washington, D.C.: 

December 28, 2001.

2000 Census: Significant Increase in Cost Per Housing Unit Compared 

to 1990 Census. GAO-02-31. Washington, D.C.: December 11, 2001.

2000 Census: Better Productivity Data Needed for Future Planning and 

Budgeting. GAO-02-4. Washington, D.C.: October 4, 2001.

2000 Census: Review of Partnership Program Highlights Best Practices 

for Future Operations. GAO-01-579. Washington, D.C.: August 20, 2001.

Decennial Censuses: Historical Data on Enumerator Productivity Are 

Limited. GAO-01-208R. Washington, D.C.: January 5, 2001.

2000 Census: Information on Short- and Long-Form Response Rates.

GAO/GGD-00-127R. Washington, D.C.: June 7, 2000.

(450133)
GAO’s Mission The General Accounting Office, the investigative arm of Congress, exists to 

support Congress in meeting its constitutional responsibilities and to help improve 

the performance and accountability of the federal government for the American 

people. GAO examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of accountability, 

integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail this 

list to you every afternoon, go to www.gao.gov and select “Subscribe to daily

E-mail alert for newly released products” under the GAO Reports heading.

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check 

or money order should be made out to the Superintendent of Documents. GAO 

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single 

address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Public Affairs Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Service Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00