1990 CENSUS ADJUSTMENT PROCEDURES

AND COVERAGE EVALUATION

HEARING

BEFORE

SUBCOMMI

CENSUS AND

| OF THE

COMMITTEE

POST OFFICE AND CIVIL SERVICE

HOUSE OF REPRESENTATIVES

NINETY-NINTH CONGRESS

SECOND SESSION

JULY 24, 1986

Serial No. 99-65

Printed for the use of the Committee on Post Office and Civil Service

U.S. GOVERNMENT PRINTING OFFICE

N

64-670 0 WASHINGTON ! 1986

For Bale by the Superintendent of Documents, Congressional Salea Office

US. Government Printing Office, Mta(«,Vi<W
/v' * '\

COMMITTEE ON POST OFFICE AND CIVIL SERVICE

WILUAM D. FOBD, Michigan, Chairman

WILLIAM (BILL) CLAY, Missouri i«-0 GENE TAYLOR, Missouri

PATRICIA SCHROEDER. Colorado ~"^ BENJAMIN A. GILMAN, New York

STEPHEN J. SOLARZ, New York CHARLES PASHAYAN, JR.. California

ROBERT GARCIA, New York FRANK HORTON, New York

MICKEY LELAND, Texas JOHN T. MYERS. Indiana

GUS YATRON, Pennsylvania DON YOUNG, Alaska

MARY ROSE OAKAR, Ohio JAMES V. HANSEN, Utah

GERRY SIKORSKI, MinnesoU DAN BURTON, Indiana

FRANK McCLOSKEY, Indiana

GARY L. ACKERMAN, New York

MERVYN M. DYMALLY, California

RON DE LUGO, Virgin Islands

MORRIS K. UDALL, Arizona

TOM DEYUUA, Staff Director

ROBERT E. LOCKHART, General Counsel

PATRICIA F. RISSLER, Deputy Staff Director and Chief Clerk

JOSEPH A. FISHER, Minority Staff Director

SUBCOMMITTEE ON CENSUS AND POPULATION

ROBERT GARCIA, New York, Chairman

MARY ROSE OAKAR, Ohio JAMES V. HANSEN, Utah

GARY L. ACKERMAN, New York JOHN T. MYERS, Indiana

LILLIAN FERNANDEZ, Subcommittee Staff Director

(II)

ri .{,d33j<f
V$y

CONTENTS

Statement of: Page

Barbara A. Bailar, Ph.D., Associate Director for Statistical Standards and

Methodology, Bureau of the Census 2

Stephen E. Fienberg, Ph.D., chairman, Committee on National Statistics,

and professor of statistics and social science, Carnegie Mellon University 29

Michael A. Stoto, associate professor of public policy, John F. Kennedy

School of Government, Harvard University 70

Carl P. Carlucci, Ph.D., executive director, New York State Legislative

Task Force on Demographic Research and Reapportionment 73

Eugene P. Ericksen, Ph.D., associate professor, Department of Sociology,

Temple University, and special consultant, National Economic Research Associates, Inc 87

Joseph Waksberg, vice president, WESTAT, Inc 103

Statement submitted by:

John W. Pratt, professor of business administration, Harvard University.. 39

Additional material:

"The Bicentennial Census: New Directions for Methodology in 1990," a

summary by John W. Pratt and Constance F. Citro 44

(in)

1990 CENSUS ADJUSTMENT PROCEDURES AND

COVERAGE EVALUATION

THURSDAY. JULY 24. lft«6

HOUSE OF REPRESENTATIVES.

SUBCOMMITTEE ON CENSUS AND POPULATION.

COMMITTEE ON POST OFFICE AND CIVIL SERVICE,

Washington. DC

The subcommittee met, pursuant to call, at 10 a.m., in room 311,

Cannon House Office Building, Hon. Robert Garcia (chairman of

the subcommittee), presiding.

Mr. GARCIA. Good morning. I welcome all of you to our hearing

today. We will be focusing on the Census Bureau's preparations for

the evaluation of the 1990 census and considerations for improving

the accuracy of the census.

When our Nation takes its bicentennial census in 1990, the most

important goal is having an accurate count of the people. Accuracy

will directly impact how fairly we calculate the number of seats

each State will have in the House of Representatives and how Federal funds will be distributed.

Unfortunately, there can never be a perfect census in which everyone is counted. The problem of undercount has been with us

since the very first census. Although the Census Bureau has made

significant advances in reducing the problem of undercount, it is

still a critical problem among urban areas and minority populations.

To compensate for the problem of undercount, some experts believe the census would be more accurate if the results were adjusted. Others argue that it will be hard to do the adjustment in time

for congressional apportionment and would greatly increase the

Bureau s discretion in determining the outcome of the count.

Today, we are privileged to have as witnesses distinguished

panels of experts who are fully familiar with the census process.

These experts will testify on the benefits and the weaknesses of ad-

{'ustment, as well as on the importance of evaluating the census. I

lope that through this hearing we can come to a better understanding of the procedures involved in improving the accuracy of

the census.

I might add that some people may question why we are having

this hearing so early; it is only 1986, and the census is approximately, 3Y2 years away. My answer to that is that we have been

down this road before, and when it comes to the question of an undercount and adjustment procedures for the census you really can't

start early enough.

(1)
My colleague, the ranking minority member of this subcommittee, will hopefully be joining us shortly, but he is at this point occupied with some very important meetings that he is attending. He

has asked me to submit for the record his opening statement, and

it will be inserted at this point.

STATEMENT BY HON. JAMES V. HANSEN

Mr. HANSEN. Mr. Chairman, I am happy to be with you today to

continue our review of the Census Bureau's adjustment procedures

and coverage evaluation.

The accuracy of census results has long been a concern. Even

though the effort to improve the accuracy of the census it is still

estimated that 1 to 2 percent of our population does not get counted. This is a problem that needs to be addressed but not surprisingly many of the suggested solutions are very controversial.

Mr. Chairman, I do have serious reservations regarding the adjustment of census figures not only because of the serious damage

it could do to the proper execution of the census but also because of

the political problems which could arise.

Our witnesses today will discuss some of the options available to

the Bureau and will hopefully provide us with information that

will enable this subcommittee and Congress to determine the best

avenue to pursue.

Mr. GARCIA. Our first panel is going to be led by Dr. Barbara A.

Bailar, who is the Associate Director for Statistical Standards and

Methodology from the U.S. Bureau of the Census. Other members

of that panel will be Dr. Steve Fienberg, who is the chairman of

the Committee on National Statistics and a professor at Carnegie

Mellon University. If Dr. Bailar and Dr. Fienberg are here, I would

appreciate it if they would be kind enough to take a seat up front

here.

STATEMENTS OF BARBARA A. BAILAR, PH.D., ASSOCIATE DIRECTOR FOR STATISTICAL STANDARDS AND METHODOLOGY,

BUREAU OF THE CENSUS, AND STEPHEN E. FIENBERG, PH.D.,

CHAntMAN, COMMITTEE ON NATIONAL STATISTICS, AND PROFESSOR OF STATISTICS AND SOCIAL SCIENCE, CARNEGIE MELLON UNIVERSITY

Dr. BAILAR. Thank you, Mr. Chairman. Thank you for this opportunity to brief the subcommittee. I will make my remarks brief" and

submit my full statement for the record.

Evaluations of coverage conducted since 1950 have shown a

steady improvement in census coverage over four decades, from an

undercount of over 4 percent for the total population in 1950 to an

undercount in 1980 of 1 to 2 percent depending on the estimated

number of illegal aliens in the country. Throughout this time there

has been a persistent difference in the undercount rate for blacks

as compared to the total population. Because there continues to be

an undercount in the census, we have embarked on a dual strategy

for the 1990 census. We will attempt to take the best census possible and count everyone, but we will also do what is necessary to be

prepared to adjust the counts if we determine that adjustment will

improve them.
Because the coverage evaluation studies of 1980 did not provide

information accurate enough to adjust the census, we have undertaken a rigorous program of research, testing and evaluation of the

various issues related to adjustment. To be able to adjust we have

to do four things:

First, we have to establish methodologies to measure coverage accurately.

Second, we have to be able to develop acceptable statistical techniques to estimate coverage for small geographic levels and for a

variety of population and housing characteristics.

Third, we have to establish and publish standards for evaluating

the quality of the adjusted and unadjusted data.

Fourth, we have to implement the adjustment, compare the adjusted and unadjusted data in light of the standards, and then release one of the sets of data as the official 1990 census results.

This is an ambitious goal. We may not be able to estimate coverage and adjust for the count and characteristics in every census

block by December 31, 1990. If we do not meet that date, we do

have fall-back positions, but it would mean that there could be two

sets of census results•the unadjusted numbers available on December 31, 1990, and an adjusted set available later. I will discuss

each of the four major steps in adjustment as well as the issue of

reaching consensus on them.

Mr. Chairman, the first issue I will talk about is the measurement of census coverage. We have tested several methods for measuring coverage and have decided on the two most promising methods: a postenumeration survey or PES, and demographic analysis.

We are conducting extensive research and testing on the design of

the PES for 1990.

First, we plan to select an independent sample for the 1990 PES

rather than piggybacking on a sample drawn for another purpose

as we did in 1980.

Second, we have more fully integrated the PES operations with

the census operations. We would be able to start PES data collection and processing on a flow basis. This early start is essential if

we are going to have any chance of completing adjustment by December 31, 1990.

Third, there are differences in the size and type of sample to be

drawn.

Fourth, we are developing an automated matching system which,

along with other new automation improvements for 1990 could give

us the ability to match the PES sample to the census more accurately and more quickly than in 1980. We have been testing these

and other improvements in our test censuses in 1985 in Tampa, FL,

and in 1986 in central Los Angeles County and in east central Mississippi.

The other principal coverage evaluation method I will mention is

demographic analysis. Demographic analysis involves the development of an estimate of the population from administrative data

records essentially independent of the census. We will develop demographic analysis estimates of the size and distribution of 1990

coverage errors at the national level. We plan to use them in conjunction with the results of the PES to measure coverage of the

1990 census.
In early 1987, we will decide on the statistical and operational

feasibility of adjustment. This is not a decision about whether the

adjusted numbers will be the official 1990 census counts. What appears feasible in 1987 may or may not be feasible in 1990. The date

for this decision is driven essentially by the timing of our budget

request for 1990.

If we determine that adjustment is not technically feasible, then

we would not go ahead with the full-scale adjustment program. We

would likely propose a smaller, less costly undercount measurement program for the purposes of evaluation but not adjustment. If

the decision is that we are statistically and operationally capable of

adjustment, then we will plan work to permit adjustment. We will

release the adjusted figures unless the final results do not meet

technical standards that we are currently developing. Although we

cannot say definitively at this time what our determination will be,

our research thus far on most aspects of undercount measurement

has been promising.

The second issue is making adjustments for small areas and for

demographic characteristics. To avoid inconsistencies in our tabulations and publications, and confusion for our data users, we plan to

adjust down to the block level and for as many characteristics as

possible.

The third issue I will discuss is the standards for deciding whether it is statistically sound to adjust. We are developing and will announce well in advance of the census the standards we will use to

assess whether the census counts or the adjusted figures are better.

We want to reach a consensus among various stakeholders•Congress, the statistical community and others•in advance of the 1990

census on the criteria for judging the relative quality of the adjusted and unadjusted figures.

We are developing standards for both methodologies of estimating census undercount•the PES and demographic analysis. These

two methodologies are very different and require different standards. The standards upon which the adjustment decision is to be

based must depend upon observable results of the census and of the

coverage measurement studies. These would be such things as the

measured differential census undercount by race, ethnicity or geographic level, indicators of the quality of the data collected in the

coverage measurement survey; and indications of the quality of the

matching operation.

We continue to conduct research into census adjustment standards and plan to distribute the results of this research in the

spring of 1987. After that we will begin an intensive schedule of

consultation with data users including Federal agencies, professional statistical organizations and the Congress.

The fourth issue I will discuss is the public review of our plans

for adjustment. We could not conduct the census without widespread support for our plans and goals. Neither can we contemplate adjustment without establishing a consensus of support, including congressional review, for our techniques and standards. We

have consulted extensively with a wide range of interested individuals and groups. This hearing and the hearing before this subcommittee 2 years ago are part of that process. We have discussed adjustment planning with professional associations, the Census Bu-
reau's advisory committees, and at our annual research conferences. And we have benefited from the recommendations of the distinguished Panel on Decennial Census Methodology of the Committee on National Statistics of the National Academy of Sciences.

After the publication of our preliminary census adjustment

standards early next year, we will hold a number of public forums

to discuss the standards and other issues related to adjustment. In

October 1987, we will publish the proposed standards in the Federal Register and review any comments we receive. And in May of

1988, we will submit materials on the proposed census adjustment

standards for review by our congressional oversight committees.

Finally, I will discuss the decision to adjust the census. There are

two main parts to the overall decision process. First, if the 1987 decision is that adjustment is statistically and operationally feasible,

the machinery will be put in place. Coverage estimates will be

made and population and housing estimates adjusted for undercount will be produced. We have stated that we will only release

the adjusted data if it can be shown that the adjustment figures

are better than the unadjusted counts. The 1987 adjustment decision will essentially represent the Census Bureau's judgment at

that time about whether to prepare adjusted estimates.

Second, in December 1990, or as soon as the coverage measurements have been obtained, we will evaluate the measurements in

light of the specified technical standards. This evaluation will be

looking for any substantial unforeseen errors that might have occurred or other unanticipated serious flaws in the coverage measurement process that would cause us to view the adjusted data as

farther from the truth than the unadjusted data.

Assuming the standards are met and such flaws have not occurred, we would release the adjusted data. But if our review indicates that serious errors occurred, then we would release the unadjusted data. To make that decision, we will convene a group of key

Census Bureau statisticians and demographers. In addition, we will

refer this matter to the Committee on National Statistics' Panel on

Decennial Census Methodology and to the Census Advisory Committee of the American Statistical Association, perhaps augmented

by some representatives selected from the Census Advisory Committee on Population Statistics. These outside experts will review

the results of the census and the adjusted figures and make comparisons with the established standards. They will submit a report

and recommendations to the internal technical group. The internal

technical group will consider the external experts' report and recommendations, and will make recommendations to the Director of

the Bureau of the Census. The Director will then make and issue

the final decision.

The plan for determining whether to adjust has been designed to

be independent of the judgment of only one individual. By developing standards that are agreed upon in advance, we are removing

the need to trust the judgment of one specific person or any one

concern. We believe that the decision must be based upon definite

knowledge about the results of our coverage evaluation program

and the quality of both the census and the evaluation. The key to

this strategy is reaching consensus on the standards before the

census is taken.
6

Mr. Chairman, that concludes my testimony on adjustment plans

for the 1990 census, and I would be happy to answer any questions.

Mr. GARCIA. Thank you, Dr. Bailar.

[The statement of Dr. Bailar follows. Also included are responses

to written questions.]
STATEMENT OF THE ASSOCIATE DIRECTOR FOR

STATISTICAL STANDARDS AND METHODOLOGY OF THE

BUREAU OF THE CENSUS

Barbara A. Ballar

Before the Subcommittee on Census and Population

Post Office and Civil Service Committee

U.S. House of Representatives

July 24, 1986

Introduction

Mr. Chairman, thank you for this opportunity to brief the Subcommittee

on the Issue of adjustment 1n the 1990 census.

Because of the many Important uses of census data, completeness and quality

have been a concern to census officials and data users since the first

census. After the 1790 enumeration. President George Washington expressed

disappointment that the population count did not exceed 4 million (1t was

3,929,214). There was no way to measure undercount then, but Washington

had a hunch that some people were reluctant to be counted.

The development of probability sampling methods, the Improvement 1n

demographic techniques, and the Improvement 1n administrative record

systems (such as birth and death registrations) 1n the middle decades of

this century have made 1t possible to provide more scientific measures of

the accuracy of census data. At the same time, there has been a growing

Interest 1n the quality of the counts, particularly for programs that use

census data to allocate funds.
8

Since 1950, each census has Included an evaluation and research program

to measure coverage error. The main purpose of these evaluations has

been to measure progress in reducing the undercounts and design programs

to correct enumeration problems 1n the next census.

These evaluations have shown a steady Improvement 1n net census coverage over

four decades, from an undercount of over 4 percent for the total population

1n 1950 to an undercount in 1980 of 1 to 2 percent, depending on the estimated

number of illegal aliens 1n the country. Throughout this time, there has been

a persistent difference 1n the undercount rates for Blacks as compared to

the total population. Although we have been able to reduce the undercount

for all groups in the last few censuses, the undercount for Blacks has

remained about 4.5 percentage points higher than the undercount for the

total population. We know less about the Hispanic population (because of

deficiencies 1n administrative records), but the data we have suggest

that this segment of the population has also been disproportionately

undercounted. Other differentials 1n undercount also exist, such as by

geographic area and socio-economic characteristics.

Census undercount might not be of such great concern If It were spread

evenly across population groups and geographical areas. But since

differential undercounts do exist, there has been considerable discussion

about their effect and possible solutions.

Since the early 1950's, some people have advocated that we adjust the

census counts to correct for the undercount. Adjustment was a major
issue for the 1980 census. As I remarked in my testimony before this

Subcommittee in April 1984, our coverage evaluation studies for the

1980 census did not provide us with information accurate enough to adjust

the 1980 census data. They provided only a general Idea of the degree of

coverage error 1n the census.

Because there continues to be an undercount in the census, we have embarked

on a dual strategy for the 1990 census: We will attempt to take the best

census possible and to count everyone, but we also will do what is necessary

to be prepared to adjust the counts if we determine that adjustment will

improve them.

The first part of this strategy 1s to attempt to count everyone. For

more than 40 years, we have been pursuing efforts to Improve the census,

and as I stated earlier, these efforts have been successful. But we also

have seen the costs of the census rise as we commit more resources to the

improvement of the data. He know that even if we had unlimited funds we

would never be able to take a perfect census in which we count everyone.

That 1s why the second part of our strategy 1s to be prepared to adjust

the census If we determine that adjustment will improve the counts. We

have undertaken a rigorous program of research, testing, and evaluation of

the various issues related to adjustment. To be able to adjust, we have to

do four things: (1) We have to establish methodologies to measure coverage

accurately; (2) we have to develop acceptable statistical techniques to

estimate coverage for small geographic levels and for a variety of population
10

and housing characteristics; (3) we have to establish and publish standards

for evaluating the quality of the adjusted and unadjusted data; and, (4)

we have to implement the adjustment, compare the adjusted and unadjusted

data 1n light of the standards, and then release one of the sets of data

as the official 1990 census results.

This 1s an ambitious goal. It means that we need to carry out the adjustment while the census counts are being tabulated. We may not be able to

estimate coverage and adjust for the count and characteristics 1n every

census block by December 31, 1990. If we do not meet that date, we do

have fallback positions; but it would mean that there could be two sets

of census results•the unadjusted numbers available on December 31, 1990

and an adjusted set available later.

In the following sections, I will discuss each of the four major steps In

adjustment, as well as the Issue of reaching consensus on them.

Coverage Measurement Methodologies

Mr. Chairman, the first issue I will talk about 1s the measurement of

census coverage. Before we can consider adjusting for an undercount, we

must be able to measure it. We would, of course, plan to measure census

coverage, as we have in the last four censuses, even if we determine

not to pursue adjustment. We have tested several methods for measuring

coverage and have decided on the two most promising methods--a postenumeration survey and demographic analysis.
11

Post Enumeration Survey

The general thrust of our research program Is toward a post-enumeration

survey (PES) as the main tool for providing detailed data about coverage

error. A PES will provide data on gross omissions and gross overenumeratlons, on subgroups of the population, and for substate areas. In a PES,

we draw a sample of the population Independently of the census. We enumerate

the sample, asking where the persons lived on Census Day and various characteristics about them. We match each of the people 1n the sample to the census

to determine whether the person was counted correctly 1n the census or missed

1n the census. In 1980, we drew our sample of about 168,000 households from

two months of the Current Population Survey (CPS). We also relntervlewed

about 110,000 households counted In the census to look for such errors as

counting someone twice, counting someone In the wrong location, or counting

someone not eligible to be enumerated under census residency rules.

Our experience with the PES has shown that there are problems with this

methodology. For example, there can be Incomplete data when enumerators are

unable to complete an Interview; there 1s difficulty in matching cases from

the PES to the census; and there 1s the lack of Independence between the PES

and the census so that the same persons are missed in both the PES and the

census. We are working to resolve these problems as much as possible, but

we do not know at this time whether we will be successful or not.

We are conducting extensive research and testing on the design of the PES

for 1990. I will now discuss our plans In four areas.
12

First, we plan to select an independent sample for the 1990 PES, rather

than piggy-backing on a sample drawn for another purpose. We did the

latter in 1980, when we used the CPS. For 1990, we will select a sample

of census blocks and enumerate every household in them. The independent

sample of blocks may be more accurate than the 1980 approach for a number

of reasons, including reduced geographic assignment errors and reduced

matching errors. In this approach we match the PES to the census and the

census to the PES, following up unmatched cases in both directions. In

1980, we had two separate surveys to measure gross omissions and gross

overenumerations with some balancing that did not work well.

Second, we have more fully integrated PES operations with the census. Our

goal will be to complete basic census field work in as many offices as possible

by July 1990. As field offices complete the enumeration, data collection for

the PES would begin. Me would be able to start PES data collection and

processing on a flow basis. This early start is essential 1f we are

going to have any chance of completing adjustment by December 31, 1990.

Third, there are differences 1n the size and type of sample to be drawn.

In 1990, we will have one sample of about 300,000 households that will give

us data on both missed persons and duplicate enumerations. In 1980, we had

two samples: one of 168,000 households to give us data on missed persons

and one of 100,000 households to give us data on duplicated persons.

Fourth, we are developing an automated matching system, which, along with

other new automation Improvements for 1990, could give us the ability to
13

match the PES sample to the census more accurately and more quickly than

1n 1980. A major problem with the 1980 PES was that the difficulty of

matching cases from the CPS to the census led to a significant number of

unresolved cases for which the enumeration status could not be determined.

This problem was not unexpected and is inevitable, to some extent, in any

matching study. Unresolved cases occur when incorrect or incomplete data

are collected in either the census or in the sample survey being matched.

It Is also Important that we have high interview rates in the PES. Nonresponse will invariably Increase the percentage of unresolved cases.

We have been conducting research on an automated matching system for some

time and have made significant progress. What makes 1t possible to use

automated matching in the census are the automation advances we plan for

the 1990 census•(1) an automated address control file, (2) early conversion

of questionnaire data Into computer-readable format, and (3) the key-entry

of names for persons 1n PES blocks and nearby blocks. While the automated

matcher may Improve matching, there will still be cases that require

clerical intervention and cases that we will not be able to match at all.

We are aiming to reduce the percentage of these cases; however, this remains

an area of concern.

We have been testing these and other improvements 1n our test censuses in

1985 in Tampa, Florida, and In 1986 1n Central Los Angeles County and in

East Central Mississippi.
14

We tested In our 1986 census In Los Angeles a variation to the PES.

This variation Involved enumerating the Independent sample before the

census, and so we call 1t a pre-enumeration survey. Matching and followup still occur after the census enumeration. We hoped that this method

would allow an earlier start to the matching and could help us meet our

timing goals; however, it 1s now uncertain that we would gain any time.

Demographic Analysis

The other principal coverage evaluation method I will mention is demographic

analysis. Demographic analysis Involves the development of an estimate

of the population from administrative data records essentially Independent

of the census. The estimated population is then compared to the census

count to measure coverage in the census. The sources used in demographic

analysis Include birth, death, and Medicare records, estimates of Immigration,

and estimates of emigration. Demographic analysis Involves the comparison of

aggregate data sets rather than case-by-case matching as 1n the PES.

Demographic analysis aims at providing national estimates of net census

error for age, sex, and race groups. Since the 1950 census, demographic

analysis has been the principal tool for estimating census coverage at

the national level. The coverage figures I quoted earlier were based on

demographic analysis. The technique does not allow the development of

estimates for substate areas because the information on net migration is

not sufficiently accurate. Demographic analysis also does not currently

allow estimation of the coverage of Hlspanics because many of the administrative records used to produce the estimates do not record whether a
15

person 1s Hispanic. (Many states now record whether a person Is Hispanic

on birth and other records, but deaographic analysis requires using records,

In soae cases, going back SO years or sore.)

One principal difficulty facing the development of deaographic analysis

estimates for 1990 1s the measurement of the undocumented alien population.

Unfortunately, there is no widely accepted estimate of undocumented aliens

in the country. Without an accurate estimate of this population, the estimate

of the total population living In the United States may be subject to greater

error than the level of undercount we are trying to measure.

We will develop deaographic analysis estimates of the size and distribution of

1990 coverage errors at the national level. We plan to use them in conjunction

with the results of the PES to measure coverage of the 1990 census.

Other Methodologies

We also have examined administrative records to determine whether they

are a useful supplement to the PES. We have examined various kinds of

lists•food stamps, driver's licenses, unemployment compensation, draft

registration. Aid to Families with Dependent Children, and so on•to

determine whether they contain large numbers of individuals from hard-tocount groups that tend to be missed in both the census and the PES. If

we determine that the lists contain a large number of such individuals,

they might be a useful supplement to the PES.
16

Another coverage measurement technique we have been Investigating is called

forward tracing. The Forward Trace Study is nearing completion. This method

utilizes an independent sample drawn from the previous census, from birth

and immigrant lists, and from lists of those persons missed in the previous

census. These samples are traced forward in time (by keeping track of any

new addresses to which persons move) to the current census, and the Individuals

are matched to the current census to determine their enumeration status. The

preliminary findings from the study are that tracing is expensive and that

there is a substantial residual group of untraced Individuals, particularly

among the demographic subpopulatlons that tend to be missed by a census.

These results led us to the decision not to use this method for coverage

evaluation for the 1990 census.

Another technique we Investigated is systematic observation. We conducted

some research 1n this area in the 1970's, and the Committee on National

Statistics' Panel on Decennial Census Methodology recommended that we conduct

research Into systematic observation for 1990. Basically, systematic or

"participant" observation Involves a trained person living or working 1n

a hard-to-enumerate area as a member of the community. The person, through

observation or inquiry, can learn how many people live In a particular

household and what their basic characteristics are. This observer's records

can be matched to the census to come up with an estimate of census coverage.

In the 1986 test census 1n Los Angeles, we have three anthropologists, each

working in an area of about 2 city blocks. All have either lived in or

have done extensive work for several years 1n the area they are observing.
17

In Addition to enumerating their assigned area, the participant observers

also are being asked to make suggestions, based on their experience, on

how we can Improve the PES interview methodology. It 1s doubtful that we

could use systematic observation on a large scale in the census. We expect

this study to be completed by the end of this year.

1987 Decision

In early 1987, we will decide on the statistical and operational feasibility

of adjustment. This 1s not a decision about whether the adjusted numbers will

be the official 1990 census counts. What appears feasible in 1987 may or may

not be feasible In 1990. It will depend on the research and testing yet to be

done and actual experience in the census.

In the course of our determination In 1987, we will examine field procedures

In the 198S and 1986 test censuses. We will review progress on the development

of a fast and accurate automated matching system and other aspects of

undercount measurement.

The date for this decision 1s driven essentially by the timing of our budget

request for 1990. If we determine that adjustment 1s not technically feasible,

then we would not go ahead with a full-scale adjustment program. We would

likely propose a smaller, less costly undercount measurement program for

the purposes of evaluation, but not adjustment.
18

If the decision 1s that we are statistically and operationally capable of

adjustment, then we will plan work to permit adjustment unless the final

results do not meet technical standards that we are currently developing.

Although we cannot say definitively at this time what our determination

will be, our research thus far on undercount measurement has been promising.

Adjustment for Small Areas and for Characteristics

The second Issue 1s making adjustments for small areas and for demographic

characteristics. To avoid Inconsistencies 1n our tabulations and publications

and confusion for our data users, we plan to adjust down to the block level

and for as many characteristics as possible. Many Important applications of

census data, such as redlstrlctlng and fund allocation, use data for small

areas. All the characteristics asked in the census have been Included

because they meet well demonstrated public needs or are required to fulfill

legal mandates or Implement governmental programs; thus, characteristics

are Important as well as total population and housing counts.

Our coverage measurement methodologies•whether case-by-case matching

techniques or demographic analysis•would provide coverage estimates only

for large geographic areas and broad demographic groups. To adjust the

census, we would need a method or combination of methods to carry these

estimates down to the local level and to adjust for characteristics. The

adjustments would be based on statistical models rather than on direct

estimates for each block.
19

In our research, we are evaluating several different approaches to carrying

the adjustment down to these levels. These Include synthetic, regression,

Bayeslan, and Imputation techniques. He will test an adjustment methodology

as part of the 1986 test census.

Standards for Adjustment

The third Issue I will discuss 1s the standards for deciding whether it 1s

statistically sound to adjust. We are developing and will announce well

in advance of the census the standards we will use to assess whether the

census counts or the adjusted figures are better. We want to reach a

consensus among various stakeholders (Congress, the statistical community,

etc.) in advance of the 1990 census on the criteria for judging the

relative quality of the adjusted and unadjusted figures.

We are conducting research to develop these standards. A major goal of

the research 1s to develop a conceptual framework to measure improvement

in census counts. Another goal 1s to develop measures of the accuracy of

the estimates of census coverage error.

We are developing standards for both methodologies of estimating census

coverage error•the PES and demographic analysis. These two methodologies

are very different and require different standards.

The standards upon which the adjustment decision Is to be based must

depend upon observable results of the census and of the coverage measurement

studies. These would be such things as the measured differential census
20

.overcount by race, ethnicity, or geographic level; Indicators of the

quality of the data collected 1n the coverage measurement survey; and

indicators of the quality of the matching operation. The determination

of how to weigh these observable results would be based upon conceptual

measures of census data accuracy.

The standards will Include an assessment of the quality of the data collected

1n the coverage measurement survey. The nonresponse rates for the survey

will be one Indicator of data quality; other indicators will be produced

by the quality control operations. The consistency among a set of alternative estimates formed when the population 1s subdivided in different

ways will provide an indication of whether the census and the survey were

as independent as they were designed to be.

The quality of the matching operation is another area the standards must

address. Indicators of the level of matching error will come from rematchfng

a sample of the cases. The percentage of cases for which a determination of

enumeration status could be made will also be an indicator of the level of

error 1n the matching operation.

The standards concerning the quality of the data from the coverage measurement survey and the matching operation are designed to provide evidence

that the assumptions underlying the methodology for estimating the census

coverage error hold. Sampling error 1s also a measure of the quality of

the survey data. We will combine the results concerning the quality of

the survey and matching operations with the sampling error to obtain a
21

range of error about estimates of census coverage error.

The standards for demographic analysis must be based on Indicators of the

quality of the underlying data and the robustness of the results 1n light

of possible variations 1n the assumptions. The data used 1n forming the

demographic analysis estimates come from several sources. Some kinds of

sensitivity analyses will be conducted. An example of such an analysis

1s an assessment of the sensitivity of the estimates of undocumented

immigration to alternative assumptions concerning the level and nature of

the mlsreportlng of citizenship and country of birth 1n previous censuses.

A conceptual framework 1s required to deal with the likelihood that

neither the census counts nor the adjusted data will be closer to the

actual population for all areas or all groups. The conceptual framework

would consist of quantitative measures of the Improvement 1n census data.

The measures of Improvement will define whether adjusted or unadjusted

data are, on balance, better.

One important issue we must address 1s how general or specific to make

the standards. The standards need to be general enough to provide flexibility in interpreting and analyzing the data. However, 1f the standards

are too general, they fail 1n their purpose of giving guidance to the

decision process.

An example of a general standard might be: "The coverage measurement

study needs to have a small nonresponse rate." A specific standard might
22

read: "The coverage measurement survey needs to have a nonresponse rate

of less than x percent."

We continue to conduct research Into census adjustment standards and plan

to distribute the results of this research In the spring of 1987. After

that, we will begin an Intensive schedule of consultation with data users

(Including Federal agencies), professional statistical organizations, and

the Congress.

Public Review

The fourth Issue I will discuss 1s public review of our plans for adjustment. Public review of our plans for adjustment 1s an essential part of our

planning process. We could not conduct the census without widespread

support for our plans and goals. Neither can we contemplate adjustment--

a complex enterprise In any case--w1thout establishing a consensus of

support (Including Congressional review) for our techniques and standards.

We have consulted extensively with a wide range of Interested individuals

and groups. This hearing and the hearing before this Subcommittee 2 years

ago are part of that process. We have discussed adjustment planning with

professional associations, the Census Bureau's advisory committees, and at

our annual research conferences. And we have benefited from the recommendations of the distinguished Panel on Decennial Census Methodology of the

Committee on National Statistics In the National Academy of Sciences.
23

We have contracted with the Academy to convene a special panel meeting 1n

the early fall of 1986 to advise on the development of adjustment-related

programs. The Academy will present a summary of observations and recommendations to us in December 1986. These recommendations will help us develop

the decennial census adjustment standards. To ensure continued participation

by the Academy, we are negotiating for a 2-year extension of the existing

contract. This contract extension will provide for on-slte consultation

with Census Bureau researchers to ensure an opportunity for the exchange

of information on a schedule compatible with our commitment to critical

milestone dates.

After the publication of our preliminary census adjustment standards early

next year, we will hold a number of public forums to discuss the standards

and other Issues related to adjustment. These discussions will be at

meetings of the census advisory committees, the annual research conference,

and special meetings and workshops. In October 1987, we will publish the

proposed standards 1n the Federal Register and review any comments we

receive. In Nay 1988, we will submit materials on the proposed census

adjustment standards for review by our Congressional oversight committees.

Decision on Adjustment

Finally, I will discuss the decision to adjust the census. Assuming that

this process of assessing technical feasibility and submitting to congressional and public review results 1n a consensus supporting the Census Bureau's

plans to adjust, the next question 1s: when do we decide to adjust and

how do we make the decision?
24

There are two main parts to the overall decision process:

First, 1f the 1987 decision 1s that adjustment is statistically and operationally feasible, the machinery will be put in place. Coverage estimates

will be made and population and housing estimates adjusted for an undercount

will be produced. We have stated that we will only release the adjusted

data if it can be shown that the adjustment figures are better than the

unadjusted counts. The 1987 adjustment decision will essentially represent

the Census Bureau's judgment, at that time, about whether to prepare adjusted

estimates.

Second, in December 1990 or as soon as the coverage measurements have been

obtained, we will evaluate the measurements 1n light of the specified

technical standards. This evaluation will be looking for any substantial

unforeseen errors that might have occurred or other unanticipated serious

flaws in the coverage measurement process that would cause us to view the

adjusted data as farther from the truth than the unadjusted data. Assuming

the standards are met and such flaws have not occurred, we would release

the adjusted data. But If our review Indicates that serious errors

occurred, then we would release the unadjusted data.

To make that decision, we will convene a group of key Census Bureau

statisticians and demographers. In addition, we will refer this matter to

the Committee on National Statistics' Panel on Decennial Census Methodology

and to the Census Advisory Committee of the American Statistical Association.

These outside experts will review the results of the census and the adjusted
figares aaa make caaaarisaas air* t*e estatOismea stataifo. "Tajy ail!

make detenrfaatiaas ami jvaaaemts aheat the accuracy of t*e anaercoan

estimates. They ai'l see if aafareseea errors ectafiem

measurement jr-ac-aa Hat aoala caase as ta viea the aaetrcoaat estimates

as inaccarate aaa. therefore, mat saitaale for ceasas acjastaeat. They

mill sabmit a reaort aaa recammemaatioms ta the iatermal technical growa.

The Internal technical araap aill consider tme external experts' reaort

and recoaaendatlons aaa aill make recaemendatioes to tme Director of the

Bureau of the Census. The Director mill then make and issue the final

decision.

The plan for determining whether to adjust has been designed to be

independent of the judgment of only one individual. By developing standards

that are agreed upon in advance, we are removing the need to trust the

judgment of one specific person or any one concern. He believe that the

decision aust be based upon definite knowledge about the results of our

coverage evaluation program and the quality of both the census and the

evaluation. The key to this strategy 1s reaching consensus on the standards

before the census Is taken.

Conclusion

Mr. Chairman, that concludes ay testimony on adjustment plans for the

1990 census. We are developing a dual strategy for the 1990 census: He

will attempt to count everyone, and we will be prepared to adjust the

census 1f we determine that adjustment will Improve the counts.
26

I want to emphasize, Mr. Chairman, that the 1987 decision 1s a decision

about what Is statistically and operationally feasible, and should not

be mistaken as a decision to adjust. What appears feasible In 1987 may or

may not be feasible 1n 1990. It will depend on forthcoming research results,

success 1n coverage improvement, and the results of coverage evaluation.

Adjustment 1s a highly complex and controversial subject. We are working

on many technical Issues that we must resolve before we can undertake

adjustment. But we must also develop an approach to adjustment that will

achieve widespread public support and understanding. Obviously, Congressional

approval and support for our adjustment plans are crucial. We appreciate

opportunities such as this to keep the Congress informed of our plans and

look forward to discussing adjustment further as our plans develop.
27

Responses to Questions from

Subcommittee on Census and Population

to

Barbara Bailar

Associate Director

Statistical Standards and Methodology

on

the Adjustment Procedures and Coverage Evaluation

July 31, 1986

QUESTION 1. What makes you believe that the Bureau will have an easier time

adjusting the census results 1n 1990? How does the situation we are likely to

face 1n 1990 differ from the one that we faced in 1980?

ANSWER: I have no expectation that the Census Bureau will have an easy time

adjusting the Census results in 1990. Coverage measurement is a complex,

highly technical task. It 1s difficult under any circumstances, and will be

no less so 1n 1990 than 1n 1980. Because of our research program, I do have

Increased expectations that the Bureau will be successful in adjusting the

1990 results. The 1980 census was not adjusted because the undercount

estimates themselves were not good enough, whereas the research to date in

preparation for 1990 leads me to have some confidence that the 1990 undercount

estimates may be accurate enough to offer an improvement upon the unadjusted

counts. We are planning for adjustment for the 1990 Census, whereas this

planning was not done in preparation for the 1980 Census. A vigorous research

program has been underway to develop improved methods of coverage measurement

and improved methods for carrying down an adjustment to local levels. As I

stated in my written testimony, progress thus far on most aspects of this

research program has been promising.

QUESTION 2. In your testimony you give quite an important role to the

National Academy of Sciences panel. At the same time, we have learned that

there have been delays in funding their work. Could you tell us, when is the

Bureau likely to make a decision about the future of the panel?

ANSWER: Our plans are to continue funding work by the panel through 1990.

QUESTION 3. What can you tell us about the results of your attempt to

evaluate the census tests in Los Angeles and Mississippi?
28

ANSWER: The coverage evaluations 1n Los Angeles and Mississippi are

proceeding well but are still at an early stage. The major field work in both

locations was completed on August 1. Special followup efforts resulted in

very few refusals, both in Los Angeles and in Mississippi. Computer matching

of the test results to the corresponding census files is scheduled to begin in

August. Final results from the Los Angeles and Mississippi evaluations will

be available by the end of the calendar year.

QUESTION 4. What will be the anticipated cost estimate if the Bureau decides

to go ahead with adjusting the census results?

ANSWER: The cost of a 1990 adjustment program will be higher than the $17

million spent for the evaluation studies in 1980. Adjustment costs are

contingent upon the methodology to be used in the original census

enumeration. Until that methodology is clarified, and that will occur later

this year, we cannot be specific about the cost estimates for coverage

measurement and adjustment in 1990. I would observe, however, that adjustment

brings three additional costs over 1980. One is a cost associated with the

increased sample size required to achieve greater geographic detail in the

coverage measurements, the second is the cost of carrying out a special

Coverage Measurement Survey rather than piggybacking on an existing survey,

and the third is the computer and clerical processing associated with an

actual adjustment of the enumeration. In recent months we have reached a

provisional estimate of the sample size required to support a census

adjustment and that estimate is around 300,000 housing units. We will refine

this estimate in the next year based upon information learned in the Los

Angeles, Mississippi, and Tampa tests about the components of sampling error

in a post enumeration survey.

QUESTION 5. If there will be no adjustment, what will be the sample size of

the census evaluation and how much will the Bureau request the Congress for

coverage evaluation?

ANSWER: If the Bureau decides in early 1987 that it is not technically

feasible to adjust the 1990 census, we will limit our work to a coverage

evaluation study. At this time we have not determined the goals for such a

program or the methodology. As a rule of thumb, I expect the cost will be on

the order of that in the 1980 Census carried forward to reflect inflation and

various other structural changes in the population.
29

Mr. GARCIA. Dr. Fienberg.

STATEMENT OF STEPHEN E. FIENBERG, PH.D.

Mr. FIENBERG. Mr. Chairman, it is a pleasure for me to appear

again before your subcommittee and to participate in this ongoing

review of the Census Bureau's plans for 1990.

The Committee on National Statistics at the National Academy

of Sciences has had special panel which has been examining the

methodology in planning for 1990. And in addition to my own testimony, I have two other documents I would like to submit for the

record: a statement by Prof. John Pratt, who is chair of our panel,

and a summary of our panel's report which was coauthored by the

panel study director, Dr. Constance Citro. She is present and prepared to answer questions that might be appropriate about the

report and its recommendations.

Mr. GARCIA. Without objection.

Mr. FIENBERG. My own comments today are divided into three

parts. The first part, briefly, deals with adjustment methodology,

the second part addresses what our panel's suggestions and recommendations were, and the third part deals with the timing of decisions.

Let me begin by noting at the outset that I am an advocate of

adjustment. I believe that the evidence marshaled by the Census

Bureau's own studies of coverage in 1960, 1970, and 1980 amply

demonstrate the existence of a substantial differential undercount,

which for the black population in 1980 was about 5 percent nationwide. It is also important to recognize that a choice not to adjust

the 1990 census would, in fact, be a choice of the method of adjustment, one which makes no change at all in the basic counts. In my

view this choice would be the wrong one because I expect that

there will be once again a differential undercount in 1990. Choosing a method that actually adjusts the counts makes more sense to

me.

Adjustment aims, by supplementing the census counts with other

information, to produce more accurate population estimates than

the raw counts themselves. As Dr. Bailar has mentioned, there are

two principal methods of adjustment that the Census Bureau has

been pursuing, and the PEP method, primarily the one that is

being focused on, is an approach which takes data from a second

source•in this case it will be a large-scale survey•and compares

the results of that source with the actual census results. The two

sources are then combined and the count of those in both the

census and the survey are used to estimate the number of individuals missed by both sources.

I am not going to try to describe in detail the methods to you,

but I would simply like to note that they are well understood and

that there is considerable agreement that they should serve as the

basis for adjustment of raw census counts. There is still the need to

sort out a variety of refinements and options within this methodology and to implement the techniques in the census context. This

sorting out and implementation is a substantial undertaking and

has been the focus of the activities of the Bureau.

64-670 0-86
30

The Panel on Decennial Census Methodology made a series of

nine recommendations on adjustment, and they are listed in an appendix to my written testimony. The panel's view of the feasibility

of adjustment for 1990 is captured by some excerpts from their

report, which I have reproduced in my written testimony, and, in

essence, what the panel is saying in those excerpts is that the Bureau's focus should be on the resolution of technical questions and

the formal implementation of adjustment operations. The panel

was focusing on the 1986 pretest at the time that report was written, and its intent was to allow for an early decision on adjustment, well in advance of the actual census.

In addition, the panel expressed concern that the Bureau might

make a decision not to adjust until it had arrived at the best possible method, even though there might be a professional consensus

that each of several methods would be superior to the use of the

raw counts. As Voltaire once said, "the best is the enemy of the

good," or as my colleague Herbert Simon would note, if a business

was making this decision, it would not necessarily choose to optimize; rather, it would choose to "satisfice".

This process of census adjustment should be viewed as part of an

overall program for coverage evaluation. Such a program provides

valuable information for the users of census data and for the

Bureau itself in its planning for subsequent censuses. Our panel

recommended that coverage evaluation plans for 1990 be broader

than those associated with this postenumeration survey methodology that is associated with adjustment.

When the panel reviewed the Census Bureau's work on adjustment, its work was really completed about a year ago, and neither

the panel nor I have had an opportunity to make an indepth followup of the progress and planning that has occurred in the interim. It is my impression that the internal research program has not

moved ahead at the pace that the panel had envisioned and that

there hasn't been as vigorous a promotion of efforts in the adjustment area as the panel would have liked. In some ways my quick

reading of Dr. Bailar's written testimony suggests that there has

been more that has gone on at the Bureau than I am aware of, and

I take great pleasure in that because I think adjustment-related research is crucial work.

The basic methodology for adjusting the census using data from

a postenumeration survey program is, as I mentioned before, well

developed and widely accepted. It is my belief that adequate procedures and agreed upon standards are available or could be developed in the near future to allow for an unambiguous decision about

adjustment in advance of 1990. I would urge that closure on the details of the adjustment methodology be reached in the near future

and that a full-scale test be made part of the census dress rehearsal. A final and reasoned decision could then be made well in advance of 1990 and the timetable suggested in Dr. Bailar's testimony

fits well with the kind of timing and process that I envision.

Both the process for decisionmaking and the standards to be used

require the outside scrutiny of professionals as well as comment

from congressional oversight committees. Any timetable for decisionmaking should be designed to allow for this kind of input.
31

It has long been my personal view that the professional statistical community could agree on a process designed to reach a decision on whether or not to adjust the census data and on detailed

adjustment methodology to be used in the event of a positive decision even though many individual statisticians have their own personal choices on how they would do adjustment. My personal

choice would involve at least two pre-enumeration rather than

post-enumeration surveys and something called multiple-capture

methodology. But I would nonetheless be willing to agree to a process that virtually excluded this multiple-survey, multiple-recapture

approach in order that there be a careful and fair evaluation of related methodologies for census adjustment.

The decisionmaking process relative to adjustment requires a dispassionate rounded discussion recognizing the full range and complexity of the technical issues. If these decisions are to be made in

a timely and open fashion, the professional consensus on process

could put the Census Bureau in what I would term a defensible position regarding the choice of methodology for 1990. Because litigation over adjustment is a drain on resources of everyone involved,

congressional support for the census decision-making process is crucial.

I would like to just take a moment and read one of the final

paragraphs in Professor Pratt's testimony because it relates to this

issue, and I think it is quite relevant. His testimony reads:

The Census Bureau is politically neutral. They should be encouraged to do the

best professional job they can, including obtaining timely outside input, with as

little fear as possible of litigation or other such adverse fallout as long as they have

proceeded professionally. One would not want their decisions about adjustment to be

swayed by a feeling, for example, that a decision not to adjust may be easier to

defend in court than a particular adjustment procedure that they would otherwise

have chosen. If there are steps the Congress can devise to help the Bureau in this

regard, I would strongly urge them.

Our panel was told something about the Australian method for certifying their

census figures. It sounded marvelous, as if the Director of the Census Bureau delivered the figures to Congress and Congress immediately passed a law saying that

these are the official figures to be used, and that was that. Maybe it is not that

simple. Maybe a different approach would be more appropriate here in the United

States and maybe nothing would work as well as the Australian method does there.

But what would work best here is a matter for your professional judgment, and not

ours.

The Committee on National Statistics and its Panel on Decennial

Census Methodology stands ready to assist the Census Bureau and

Congress in decisionmaking on the 1990 Census in the years to

come. Thank you.

Mr. GARCIA. Thank you very much.

[The statements of Mr. Fienberg, Mr. John W. Pratt, and the

summary referred to follow:]
July 24. 1986

STATEMENT

OF

STEPHEN E. FIENBERG

MAURICE FALK PROFESSOR OF STATISTICS AND SOCIAL SCIENCE AT

CARNEGIE MELLON UNIVERSITY

TO THE

SUBCOMMITTEE ON CENSUS AND POPULATION

OF THE

COMMITTEE ON POST OFFICE AND CIVIL SERVICE

US. HOUSE OF REPRESENTATIVES

Mr. Chairman, it is a pleasure for me to appear before your Subcommittee again,

and to participate in your ongoing review of the Census Bureau's plans for the 1990

Decennial Census. I have been following this planning effort with great interest, in

part because of my personal research activities, and in part through my position as

Chairman of the Committee on National Statistics, at the National Academy of

Sciences. The Committee has had a special panel, commissioned by the Bureau of

the Census, which has been examining the methodological planning for 1990. The

Panel issued a report last fall entitled: The Bicentennial Census: New Directions for

Methodology in 1990. and several of the Panel's comments and recommendations are

on the issue of adjustment, which is the focus of today's hearing.

In addition to my own testimony I have two other documents I would like to

submit for the record, a summary of our Panel's report and a separate statement by

Professor John W. Pratt, of the Graduate School of Business at Harvard University.

Professor Pratt served as Chair of our Panel on Census Methodology and was unable

to attend today's hearing. Dr. Constance F. Citro, the Panel's study director and the

co-author of the summary, is present and is prepared to answer any questions you

might have about the report and its recommendations.

My comments today are divided into three parts, the first part is a brief history

of the capture-recapture technique which forms the cornerstone of most proposals

for adjustment methodology; the second part addresses the Panel on Census

Methodology's evaluation of approaches to adjustment and criteria to be used in
33

making a decision on whether or not to adjust; and the third part focusses on the

timing of decisions on adjustment.

As a prelude to these comments let me note that I am an advocate of adjustment.

I believe the evidence marshalled by the Census Bureau's own studies of coverage in

1960. 1970. and 1980 amply demonstrate the existence of a substantial differential

undercount. which for the black population in 1980 was about 5 percent nationwide.

It is also important to recognize that a choice not to adjust the 1990 Census would be

a choice of a method ot adjustment, one which made no change in the basic Census

counts. In my view this choice would be the wrong one because I expect that there

will be an undercount again in 1990. Choosing a method that actually adjusts the

counts makes much more sense.

METHODOLOGY FOR ADJUSTMENT

Adjustment aims, by supplementing the census counts with other information, to

produce more accurate population estimates than the raw counts themselves. Two

principal methods of adjustment have been used by the Census Bureau in its own

studies of undercount • the demographic method (which is applicable only at the

national level and which requires estimates of the population of undocumented aliens)

and the post-enumeration program (PEP) method (which is the primary approach that

has been proposed for adjusting the 1990 Census).

The PEP approach takes data from a second source, such as the Current Population

Survey, and compares this source with actual census results. The two sources are

then combined and the count of those in both the census and the survey is used to

estimate the number of individuals missed by both sources. The basic methodology

for making such estimates usually goes under the label, capture-recapture, because of

its original use in the study of biological populations, as long ago as 1895. Capturerecapture methods have been used in the study of human populations beginning in the

1940 s. and much of it goes under the heading of dual system estimation. There is

now an extensive literature on the topic which includes fine contributions by

statistical researchers at the Census Bureau itself. In particular, I note the work of
34

Kirk Wolter on adapting the traditional approach to the sample survey setting, and the

recent paper by Charles Cowan and Donald Malec on capture-recapture when both

sources have clustered observations.

My purpose today is not to describe these methods for you. but to note that they

are well understood and that there is considerable agreement that they should serve

as the basis for adjustment of raw census counts. There is still the need to sort out

various refinements and options within this methodology and to implement these

techniques in the census context. This sorting out and implementation is a

substantial undertaking and has been the focus of ongoing research activities at the

Bureau.

RECOMMENDATIONS OF THE PANEL ON DECENNIAL CENSUS METHODOLOGY

It was against this background that the Panel on Decennial Census Methodology

reviewed the Bureau's planning in the adjustment area. The nine recommendations on

adjustment In Chapter 7 of the Panel's report (and listed at the end of this testimony)

are focussed on adjustment to minimize differential coverage errors. As we review

them today, it is important to remember that they were written in the spring of 1985.

The Panel's view of the feasibility of adjustment in 1990 is well captured by the

following excerpts from its comments:

Many technical questions remain to be answered if adjustment procedures

are to be developed in time for their use in the 1990 census. On the whole.

while much effort will be required, the panel is optimistic that substantial

progress can be made, and many feel that this progress could well be

sufficient to permit adjustment to become a feasible and desirable part of

the 1990 census process.

We note that there are several different methods of adjustment that have

been suggested so far, and we anticipate that others will be proposed. It is

possible that a variety of alternatives, including compromise possibilities.

will be developed with evidence that each would be an improvement over

the census count, but with no obvious basis for choosing among them. In

our view, this situation should not by itself preclude the Census Bureau from

making adjustments and picking one of the alternatives.

In essence, the Panel noted that the focus of the Bureau's program in the

adjustment area should be on the resolution of technical questions and the formal

implementation of adjustment operations in a 19B6 pretest to allow for an early
35

decision on adjustment, well in advance of the actual census in 1990. In addition,

the Panel was concerned that the Census Bureau might make a decision not to ad|ust

until it had arrived at the "best" method, even though there might be a consensus

that each of several methods would be superior to the use of the raw census counts.

Census adjustment should be viewed as part of an overall program for coverage

evaluation. Such a program provides valuable information for users of census data

and for the Bureau itself in its planning of subsequent censuses. Despite the

pressure from the outside to focus on adjustment-related methodology, the Census

Bureau and Congress should not lose sight of the need for a well-conceived coverage

evaluation program. The Panel also recommended that the coverage evaluation plans

for 1990 be broader than those associated with the post-enumeration (or possibly

pre-enumeration) survey methodology associated with adjustment. As I mentioned in

my testimony before this Subcommittee in May. substantial effort needs to go into

the planning for this program in order thai the Bureau can adequately evaluate census

coverage prior to the deadline for reporting apportionment figures in 1990.

THE TIME FRAME FOR DECISION-MAKING

The Panels review of the Census Bureaus work on adjustment methodology was

completed over a year ago, and neither the Panel nor I have had an opportunity to

make an in-depth followup of the progress and planning that has occurred In the

interim. It is my impression, however, that the internal Bureau research program has

not moved ahead at the pace envisioned by the Panel and that the Bureau has not

promoted and supported, vigorously enough, related statistical research in the

academic statistical community on the technical problems involved in adjustment

methodology. Moreover, the Bureau appears to have delayed the testing of

alternative adjustment methods and deferred the timing of various decisions and

choices.

A report prepared internally at the Census Bureau, subsequent to the completion of

the Panel's first stage activities, suggested that a decision on whether to adjust be

deferred until after all of the figures are available. In my view such a plan is sheer
36

folly, and Is designed to ensure a last-minute decision not to adjust the raw counts.

Equally unacceptable to me, and I suspect to the members of the Panel on Decennial

Census Methodology, would be an early decision not to adjust because the Bureau is

unable to decide upon and implement an operational adjustment procedure.

The basic methodology for adjusting the census using data from pre-enumeration or

post-enumeration surveys is well-developed and widely-accepted. It is my belief that

adequate procedures and agreed-upon standards are available or could be developed

in the near future in order to allow for an unambiguous decision about adjustment in

advance. I would urge that closure on the details of adjustment methodology be

reached in the near future and that a full-scale test be conducted in conjunction with

the census dress rehearsal. A final and reasoned decision could then be made well

in advance of the 1990 Census. Both the process for decision making and the

standards to be used require outside scrutiny and comment as well as congressional

oversight and guidance. Any timetable for decision making should be designed to

allow for this input.

It has long been my personal view that the professional statistical community

could agree on a process designed to reach a decision (a) on whether or not to

adjust the census data, and (b) on the detailed adjustment methodology to be used in

the event of a positive decision, even though individual statisticians might differ in

their personal choices. For example, my personal choices of methodology involve

two or more pre-enumeration surveys and multiple-recapture methods for estimation.

Nonetheless. I would be willing to agree to a process that virtually excluded the

multiple-survey, multiple-recapture approach provided that a careful and fair

evaluation of related methodologies were to occur.

The decision-making process relative to adjustment requires a dispassionate

rounded discussion recognizing the full range and complexity of the technical issues.

If these decisions are made in a timely and open fashion, the professional consensus

on process could put the Census Bureau in a defensible position regarding the choice

of methodology for 1990. Because litigation over adjustment is a drain on the
resources of everyone involved, congressional support for the census decision-making

process is crucial. The Committee on National Statistics and its Panel on Decennial

Census Methodology stand ready to assist the Census Bureau and Congress in this

decision-making process in the coming years.

APPENDIX RECOMMENDATIONS OF THE PANEL ON DECENNIAL CENSUS

METHODOLOGY REGARDING ADJUSTMENT OF CENSUS DATA

Recommendation 7.1. Completeness of the count is an important goal, both for

ensuring the accuracy of the census and for establishing the credibility of the census

figures among all users. Adjustment should not be viewed as an alternative to

obtaining as complete a count as possible through cost-effective means.

Nevertheless, the ultimate goal is that of the accuracy of the published figures.

Given the likelihood that the census will continue to produce different rates of

undercoverage for various population groups, and given the equity problems caused

thereby, we recommend that work proceed on the development of adjustment

procedures and that adjustment be implemented if there is reasonable confidence that

it will reduce differential coverage errors.

Recommendation 7.2 In measuring the total loss associated with an adjustment

procedure, we recommend that the contribution to this loss attributable to a

geographic region should reflect its population size. Thus, we recommend against

loss functions based solely on the number of political entities losing or gaming

through adjustment.

Recommendation 7.3. We believe that, in general, the results of an adjustment are

likely to be affected more by the quality of coverage evaluation data and the models

and methodology used than by the choice of loss functions. Given a family of loss

functions with relatively similar objectives, it should be possible, and desirable, to

determine an adjustment procedure that has good performance for most • or all of

them. We recommend that the Census Bureau investigate the construction of

adjustment procedures that are robust to a reasonable range of loss functions.

Recommendation 7.4. We recommend that the Census Bureau explore methods for
38

providing estimates of errors associated with estimates of census over- and

undercoverage. with a view to publishing such error estimates along with coverage

evaluation results and any adjusted census data that may be issued.

Recommendation 7.5. The panel believes that it is important to strive for internal

consistency of published census figures. Should adjustment appear feasible and

effective, methods exist for distributing adjusted totals for aggregated groups down

to subgroup values. We recommend that one of these methods be used to achieve

internal consistency of census figures.

Recommendition 7.6*. Census data used for reapportionment and redistricting are

required by law to be produced no later than specific dates. It is possible that

adjustment of the 1990 census will prove feasible and effective in all respects,

except for the ability to meet the required deadlines. This should not necessarily

preclude subsequent issuance of adjusted data for other uses. In this situation, we

recommend that the Census Bureau seek determination by Congress of whether it

desires that adjusted data be used and will therefore extend the deadlines, or wishes

to adhere to current deadlines and will therefore stipulate the use of unadjusted (or

partially adjusted) data for reapportionment and redistricting.

Recommendation 7.7. The panel recognizes that considerable work is still necessary

and likely to lead to improved procedures for adjusting census data. We therefore

support the Census Bureau's stated plans to pursue, internally, research and

development of adjustment procedures, and we also recommend that the Census

Bureau vigorously promote and support related statistical research In the academic

community.

Recommendation 7.8. The panel supports the Census Bureau in its plans for a 1986

pretest of adjustment operations, including the production of mock tabulations of

adjusted census data. We recommend analysis of the resulting adjusted and

unadjusted data sets, to help identify the strengths and weaknesses of the particular

methods tried.

Recommendation 7.9. We recommend that research on adjustment include: (1)

investigations of the assumptions underlying the procedures. (2) an attempt to

evaluate empirically the more important of the assumptions as well as the sensitivity

of methods to violation of assumptions. (3) study of methods used for carrying down

estimates to lower levels of aggregation, and (4) a study of the impact of adjustment

on uses of census data.
July 24, 1966

swoon

OF

JCHi Vi. BAIT

PROFESSOR OF BUSDESS ABOKISTSAIIOK

HARVARD •am

ID IK

SIBC0HGTTEE OK CEXSJS AST POPUXnON

OF THE

commxE OK POST OFFICE AJB CT.IL SERVICE

U. S. HOUSE OF REPRESENTATIVES

I am sony not to be present in person. The problem under discussion

• adjusting the Decennial Census • is extremely important and difficult.

This was a major reason for the commissioning of a National Research Council panel and for my willingness to chair it as a nonparticipant in the

debate and litigation. I certainly support all we said in our report, and

would personally perhaps emphasize even more pressing on with the research

and development work needed before an acceptable adjustment. I have read

Professor Fienberg's statement of July 24 and concur with its thrust over

all. Regarding the chance of being ready to carry out an adjustment in

1990 if we really try, I would be more optimistic than the report sounds,

but not quite as optimistic as Professor Fienberg's statement sounds. 1

would like to make a few comments, which may add something to these documents, but will not be as well organized as either of them • as may already

be evident.

Adjusting Decennial Census data, especially for undercount, is a

large and complex problem. Vast funding and planning efforts are significantly affected by the adjustment chosen, which is inevitably an estimate,

be the estimate 0 or the simplest kind of "capture-recapture" ratio adjustment or the result of an elaborate procedure based on a model of how

the probability of being missed relates to the characteristics of individuals
40

or groups. So much hangs on the numbers that the discussions among statistical experts about the choice of procedure is 1980 seem to have been

displaced largely to the courts, an extremely cumbersome, expensive, divisive, and inappropriate forum for what ought to be an open, friendly,

professional exchange of technical judgments without political or legal

overtones.

Another reason the discussion went to the courts is that it is hard,

even for professionals, to recognize and face up to the fact that a "complete" count needs adjustment, and hard to face up to explaining it to the

public. It may seem trivial to count people, but it isn't. You want to

include small children, travelers, students away from home, street people,

drifters, and illegal aliens; people in hospitals, nursing homes, mental or

penal institutions, military service, vacation homes, and hotels; people on

welfare who fear their payments may be affected; and so on. You want to

count them once and once only, in the correct household and location, with

correct characteristics. Some people are answering for others, with possible errors and duplication. In fact, once you start to think about doing

a Census in the field, from Bedford-Stuyvesant to Berkeley, with a nationwide field staff recruited for a few months' work every ten years, you have

no trouble realizing how difficult it is, and how impossible to make it

perfect, even if resources were much lar.er than they are.

The Census Bureau has set very high standards, and has been quite remarkable in its willingness to evaluate its own work and its honesty in

doing so. Its own evaluations were the main source of information on the

undercount in 1960. Perhaps the Bureau would have avoided much heat,

stress, and time in court if it had never done anything to evaluate its own

work. It is important that it have the encouragement, mandate, and resources

to carry out evaluations in the future, both the gathering and the analysis
41

of evaluation data. In particular, the natural tendency to shortchange

analysis under pressure of new business and budgetary stringency needs to

be resisted.

Adjustment and evaluation have much in common intellectually, but

neither is a substitute for the other operationally. Adjustment calls for

an entirely different scale, detail, and tuning. One approach is to trace

people forward from the previous Decennial Census, subtract deaths and

emigration, add births and immigration, and compare the result to the current Decennial Census. In the United States a satisfactory adjustment

procedure cannot be arrived at this way, because of our great mobility,

lack of emigration records, inadequate records on legal immigration, illegal inmigration, and so forth.

Another thought is to do a perfect census in a sample of places and

infer the correction needed elsewhere. But the best judgment of those who

know is that even the best staff in a sample of places cannot achieve perfection. Furthermore, the inferential problem is not much easier or freer

of assumptions than for capture-recapture methods. In fact if the sample

is taken separately from the census it is just the extreme case of perfect

recapture, while a sample of places treated differently within the census

would provide very weak information for correcting the standardly enumerated places.

Imagine you did the whole census twice and discovered which people

were counted ("captured") both times, which were counted the first time but

not the second, and which were counted the second time but not the first.

You want to estimate the number of people not counted either time, and the

age, sex, race, location, etc., of each. If everyone had the same chance

of being missed in a census, it would be simple: the chance could be estimated (very accurately if the census is large) by the fraction of people
42

counted in the first census but missed in the second (or, for that matter,

by the fraction counted in the second but missed in the first). Knowing

this chance, one would know how many people there are altogether compared

to the number counted.

Of course the chance of being missed is different for different

types of people • that is what the differential undercount is. What the

chance is in different groups can be estimated by looking at each group

separately. But if the groups are too small, the estimates will not be

very accurate. So the undercount of black male centenarians in Berkeley

will be hard to estimate. And since we cannot afford two censuses, one

must be replaced by a sample. This means that many categories of interest

will be too small to look at individually, and we will have to make use of

collateral information to estimate the undercount in each category. This

requires some assumptions about what collateral information is relevant to

each category, and how it relates. The assumptions will be implicit if

not explicit, and partly but not totally checkable. And there are tradeoffs between improved accuracy if assumptions are correct and inaccuracy

if they are not. So now it is getting complicated and it is time to leave

it to the experts.

All methods depend on matching • determining whether someone listed

on one occasion was also listed on another, despite possible variations in

name, location, and characteristics and lengthy lists. Estimating and allowing for the inevitable imperfection in matching is another complicated

subject for the experts.

So adjustment requires a lot of difficult work, it can't be done

perfectly, and it depends on expert judgment. But the census is a lot of

work and can't be done perfectly. And one judgment that seems clear to me
43

is that on unadjusted census is not the best we can do if we are willing

to devote adequate resources to adjustment. There will always be some adjustment procedures that are worse than nothing, and the best adjustnent

can never be determined unequivocally, but there must be some procedures

that are better than nothing, and we should be able to develop and assent

to one.

This brings me back to the problem of litigation over the Census.

It is my impression that many of the Census Bureau's best people spent

major parts of their time for several years dealing with suits and

threatened suits over the 1980 census, at great cost to other activities.

There must have been a similar cost to some of the governmental units bringing suit, and substantial court and legal costs. While some of the

professional discussion engendered was useful, this is the most inefficient way to engender it that 1 can imagine, and the adversarial atmosphere

seriously slows, hampers, and distorts it besides, I believe.

The Census Bureau is politically neutral. They should be encouraged

to do the best professional job they can, including obtaining timely outside input, with as little fear as possible of litigation or other such

adverse fallout as long as they have proceeded professionally. One would

not want their decisions about adjustment to be swayed o> a feeling, for

example, that a decision not to adjust may be easier to defend in court

than a particular adjustment procedure they would otherwise have chosen.

If there are steps the Congress can devise to help the Bureau in this regard, I would strongly urge them.

Our panel was told something about the Australian method of certifying their census figures. It sounded marvelous • as if the Director of

the Census Bureau delivered the figures to Congress, Congress immediately

passed a law saying that these are the official figures that will be used,

and that was that. Maybe it is not that simple, maybe a different approach

would be more appropriate here in the United States, and maybe nothing

would work as well here as the Australian method does there. But what

would work best here is a matter for your professional judgment, not mine.

I am grateful for the opportunity to present this statement.
44

The Bicentennial Census:

New Directions for Methodology in 1990

Summary

John W. Pratt and Constance F. Citro

March 1986

This summary represents an abridged and reorganized version of Chapter 1 of the report of the Panel on Decennial Census Methodology, The Bicentennial Census: New

Directions for Methodology in 1990 (Constance F. Citro and Michael L. Cohen, eds.,

Washington. D.C.: National Academy Press, 1985). The authors wish to acknowledge

the vital contributions to the report of the members of the panel and of Michael L.

Cohen, who served as research associate for the panel. The project was supported by

funds from the Bureau of the Census, U.S. Department of Commerce.
45

PANEL ON DECENNIAL CENSl'S METHODOLOGY

John w. Pratt (Chair), Graduate School of Business, Harvard University

'Pastor* San Juan Caffrrty, School of Social Service Administration, University of

Chicago

*Anslcy J. Coale. Office of Population Research, Princeton University

Donald R. Deskins. Department of Sociology. University of Michigan

l>an P. Fellegi. Statistics Canada. Ottawa

Wayne A. Fuller. Department of Statistics. Iowa State University

"Omer R. Galle. Population Research Center, University of Texas

Joseph B. Kadane. Department of Statistics, Carnegie-Mellon University

Benjamin F. King, Educational Testing Service. Princeton, N.J.

Albert Madansky, Graduate School of Business. University of Chicago

'Alberto Palloni, Center for Demography and Ecology. University of Wisconsin

John E. Rolph. The RAND Corporation, Santa Monica, Calif.

Courtenay M. Slater. CEC Associates, Washington. DC.

"Michael A. Stoto. JFK School of Government, Harvard University

Joseph Waksberg, Wcstat, Inc.. Rockville, Md.

Constance F. Citro. Study Director

Michael L. Cohen, Research Associate

Agnes E. Gaskin, Administrative Secretary

'Member of the panel January 1984 • December 1985

"Member of the panel March 1986 - present.
46

1. Introduction

The Panel on Decennial Census Methodology of the U.S. Committee on National Statistics recently completed a review of census practices and procedures in the United

States and made a series of recommendations directed to the planning of the nation's

bicentennial census in 1990. As charged by its sponsor, the U.S. Census Bureau, the

panel investigated several new directions Tor census methodology to address the growing challenge posed by societal needs for small-area data that meet high standards of

quality yet are produced at reasonable cost. In this paper we summarize the panel's

thinking and recommendations in its report. The Bicentennial Census: New Directions

for Methodology in 1990 (Citro and Cohen, 1983). These recommendations are timely

because the U.S. Census Bureau has under way an extensive program of research and

testing for the 1990 census.

2. Census-Taking In the United States of America

Periodic censuses of population are a long-established tradition in the U.S., with roots

going back to the earliest years of the colonial period. The royal colony of Virginia

conducted the first census in North America in the early seventeenth century, and censuses of Individual colonies were frequently attempted during the colonial era (U.S.

Bureau of the Census, 1970, p. 3).

Political necessity led to the requirement for a periodic complete enumeration of

the population in the new nation formed after the American Revolution. In the compromise between large and small states made at the 1787 Constitutional Convention, the

delegates voted to provide equal representation for each state in the Senate and representation proportional to population in the House of Representatives; the population of

each state was to be determined through a decennial census. Article I, section 2, of

the Constitution stipulates:

Representatives and direct Taxes shall be apportioned among the

several Stales which may be included within this Union, according to

their respective Numbers. . . . The actual Enumeration shall be made

within three Years after the first Meeting of the Congress of the

United States, and within every subsequent Term of ten Years, in such

Manner as they shall by Law direct.

Although fundamental issues of the structure of government provided the motivation for the U.S. decennial census, the country's leaders recognized from the beginning

that the census could be a valuable source of information for many other purposes.

James Madison noted in 1789 (U.S. Bureau of the Census, 1970, p. 4) that Congress:

had now an opportunity of obtaining the most useful information for

those who should hereafter be called upon to legislate for their country, if this bill was extended to embrace some other objects besides

the bare enumeration of the inhabitants: it would enable them to adapt

the public measures to the particular circumstances of the community.

The first census in 1790 asked the age, sex, and race of each resident. During the

next 100 years, the census became firmly established as an important information

resource. The centennial census in 1890 asked questions on more subjects than any

census before or since, including 30 items on the basic population questionnaire, several housing questions, and special inquiries about decedents, inmates of almshouses and

prisons, Indians on and off reservations. Civil War veterans and widows of veterans.
and several categories of aaearalh aad physically disabled people ,li Bereaa of the

Census, 1973. pp. T441X

Work B mam mmma way aa ptaa far tfce aatioa's tuceatcaatal ceasas of popvtatw*

and hoaxing, acaedaled to take peace oa April 1. 1990. Reflecting a loag-uaadiag

tradition of iaannaaanaaat aad audificauoa to awct rhi aging iaforauuoa needs aad to

take advantage of technological advances. Cessna-taking ia Ike tweatictk ccatary kat

come to differ ia aii< important laaaatal front census-taking ia the aiaciccatk century. Some fcatarcs that have beea introduced into modern L\S- censuses aad »ill

undoubtedly coatiaac ia 1990 are:

o Since 1910. the census has beea directed by a permanent organization, the IS

Bureau of the Census. with aa experienced, professional staff in charge of

planning aad supervising the operation.

o Since 1940. statistical sampling methods have beea used to obtain responses to

many census items, so that a large volume of useful information can be

gathered without placing the burden on every household of responding to all

questions (the 1980 census asked 7 population and 12 housing items of all

households, while about 20 percent of households were asked an additional 26

population and 20 housing questions).

o Since 1970. the L" S Postal Service has delivered most of the census

questionnaires, and households asked to mail their completed questionnaires lo

census offices. Enumerators telephone or visit only those households that do

not completely respond i95 percent of households were sent questionnaires by

mail in 1980 and 83 percent of them returned their questionnaires by mail).

o Since I960, large computers have been used to process the census returns in a

relatively short span of time; in contrast, the 1890 census required almost a

full decade to process, even with the introduction of punchcard machines to

help the clerical work force.

o Since 1930, intensive effort has been devoted to evaluating the completeness of

coverage of the total population and of important subgroups and geographic

areas.

Undoubtedly the 1990 census will also differ from the most recent censuses in the

United States. Most of the differences are likely to represent incremental improvements and modifications to tried and tested procedures: for example, mailout-mailbick

techniques may be extended to the remaining 5 percent of the population residing in

sparsely settled rural areas that enumerators personally canvassed in 1980. But pttlsures are growing in this country, as in other Western nations, to address the problem!

of rising costs of traditional census practices on one hand and to satisfy expressed

needs for greater accuracy in the numbers on the other. Consequently, exploration of

changes in methods and techniques that mark a greater break with tradition Is undtr

way: for example, one proposal that has received much attention is the UH of

statistical techniques to adjust the field counts for deficiencies in the enumeration.

Major changes in census methodology, such as the use of sampling for content and

mailout-mailback enumeration, have often been made on a small scale in one consul and

then more fully implemented in the next. The 1990 census will be part of a continuing

evolution that may lead to a methodology in the twenty-first century (hat diffcri ai

significantly from current methodology as current methodology differs from thai of Ihe

nineteenth century.
48

3. The Planning Cycle for 1990

Planning for the 1990 census officially began in fall 1983 with an appropriation for

fiscal 1984. Well before that date, however, substantial work of direct relevance for

1990 was conducted. The 1980 decennial program included several experiments and

post-enumeration studies designed to help plan improvements in methodology for subsequent censuses. Pretests carried out in the late 1970s of concepts and procedures

considered for 1980 also had results that are useful for 1990 census planning.

To the general public and many casual users of census data, it may appear that the

Census Bureau has ample time to plan wisely for the 1990 census, given the start of

the planning process more than six years prior to Census Day, April 1, 1990, and the

foundation of research already completed in connection with prior censuses. In fact,

as a review of the Census Bureau's field test schedule for 1990 indicates, there are

relatively few opportunities to test thoroughly changes or modifications in census

procedures, particularly if the changes represent major departures from the past.

Moreover, evaluation of the likely impact of important changes is hampered by the fact

that pretests cannot adequately assess the effects of alternative procedures on public

cooperation with the census•only tests conducted under census conditions, that is,

experiments incorporated into an actual census as distinct from pretests, can fully

address this important question.

The Census Bureau's 1990 census testing program began in spring 1984 with tests

of mailing list compilation methods in several localities around the country i.L'S.

Bureau of the Census, 1984). Two large-scale pretests were fielded in spring 1985.

Pretests will also be conducted in 1986 and 1987. Finally, the research and testing

program will culminate in 1988 in "dress rehearsals" of the procedures planned for

1990. This schedule not only compresses into a few years the opportunities to test

new methodology but also compresses the time available to evaluate the results from

one test and incorporate them into the design of the next.

In addition to the compressed time schedule for testing and research, two other

critical factors affect the ability of the Census Bureau to modify census methodology:

staff and budget resources. The Census Bureau has long been known for the high

quality and dedication of its technical staff. The current budget for research on decennial census methodology, particularly for research on the undercount, is large by

the standards of earlier censuses. Nevertheless, no agency of government, particularly

in the constrained world of the 1980s, can expect to have sufficient staff or resources

to try out more than a few promising ideas and concepts. Pressures in the next few

years to reduce the federal government's large deficit may make it more than usually

difficult to obtain adequate staff and funding to carry out a thorough research and

testing program for 1990. Hence, it is critical to designing the best census for 1990

and to being in the best position to plan further design changes for 2000 that the

Census Bureau make the most of the testing opportunities afforded over the next few

years and establish priorities for testing and research wisely.

4. The Importance of Choice of Methodology for 1990

Controversy surrounding population censuses has as long a history in the United States

as census-taking itself. According to one review (Bureau of the Census, 1982a,

App.IIIb, p. 73), censuses conducted during the colonial period, generally at the

direction of the Privy Council or the British Board of Trade, "were seldom regarded as

complete or successful, as people perceived them being for the purposes of taxation or

conscription and were evasive and uncooperative." The decennial censuses conducted in

the new nation had a constitutional mandate according them legitimacy and support.

Moreover, Conk (1983, p. 7) has noted that: "After the first few censuses, Americans
49

became increasingly interested in the census results . . . [which) showed that the population was growing steadily and extremely rapidly." It quickly became evident in the

early nineteenth century, however, that not all areas were sharing equally in population

growth and that reapportionment based on census results meant substantial shifts in

political power. Conk (1983. p.8) continues:

/( ft not surprising therefore that nineteenth century Americans who

were pleased with the overall thrust of population change claimed that

the census proved the virtue of the American way of life or the

American system of government. Conversely. those who felt

shortchanged by reapportionment or were concerned about the tendencies of population change challenged both the census and the

apportionment system.

The first extensive criticism of the census by statisticians occurred in 1843 when

the American Statistical Association (ASA) issued a lengthy report that documented

glaring errors in the data on education, occupation, and especially the classification by

race of persons identified as insane, idiotic, and deaf and dumb. The ASA recommended that these results should be corrected or, at the least, disavowed. Problems

with both undcrcount and fraudulent additions to the count were documented in many

early censuses (U.S. Bureau of the Census. 1982a. App.IIIb. pp. 81-83).

Congress did not as a general rule respond directly to these criticisms, although

occasionally it acted to alter the apportionment of the House when there was strong

evidence of gross deficiencies in the population count. Congress gave a third

representative to Alabama in 1823 when the claim was made that the 1820 census omitted two counties and in I860 awarded an additional seat to California because of

problems with the census in that state (U.S. Bureau of the Census, 1982a, App.IIIb, p.

82). These actions were politically much more palatable than similar actions would be

today, because reapportionment legislation up through 1910 added representatives to

accommodate population growth rather than allocating a fixed number of seats among

the states.

Despite the questions raised about the population enumeration in the past, a review

of decennial census history suggests that social and political forces have converged in

recent years to make the census in this country•and in other countries as well•a

matter of demonstrably greater controversy than before. Several factors are involved.

On one hand lie increased concern with the need to protect the privacy of individual citizens and a sense that the public is ovcrsurveyed and less willing to respond to

government inquiries. Indeed, in the last few years, the level of public suspicion and

hostility to plans for the census caused the governments of several Western European

countries to delay their census programs or cancel them entirely (see Butz, 1984;

Redfern. 1983).

On the other hand, legislators have increasingly turned to statistics in making

tough policy decisions. In fiscal 1984, federal grant-in-aid programs allocated at least

S80 billion to states and localities via formulas that depended in important ways on

census figures (or statistics based on census figures, such as current population

estimates) to determine which areas received how many dollars (U.S. Office of Management and Budget, 1985). As noted above, census data are used by constitutional mandate to determine the number of scats in the U.S. House of Representatives that are

allotted to each state. They are used as well in drawing up congressional and state

and local legislative districts to meet rigid criteria for equitable representation of the

population. Data requirements for redistricting purposes in 1980 included census tabulations of the population by race and Hispanic origin for each of several million city

blocks in urban parts of the country and enumeration districts in unblocked areas (US.

Bureau of the Census. 1982b, p. 79).
50

In addition to these critical governmental needs, census data support many other

major uses. Data from the latest census serve to document the social and economic

condition of the country as a whole and are the single most important source of information for small areas and groups in the population. Comparative information from

successive censuses illuminates trends over time. Researchers, planners, and decision

makers in business, government, and academic institutions make use of census data for

a wide range of important planning and analysis purposes. Just a few of the many

uses to which census data are put include:

o Site selection for public service facilities and commercial establishments based

on evaluating the socioeconomic characteristics of alternative locations;

o Transportation planning using detailed data on commuting flows; and

o Research into changing rates of population growth for metropolitan versus

nonmctropolitan areas and different regions of the country.

Many analyses based on census data have implications for the distribution of political power and wealth among various population groups in the country. For example,

census data on the racial, ethnic, age, and sex makeup of occupational groups in labor

market areas are used to assess the extent to which work forces reflect the

characteristics of the local labor force. These data frequently form the basis of

antidiscrimination lawsuits brought against employers. Census data on the makeup of

the local population are used to assess•and challenge•the representativeness of grand

and petit juries. Census data on earnings cross-tabulated by various characteristics are

used to analyze wage disparities within and among occupations and important

population subgroups. Findings from such studies can affect outcomes of public policy

deliberations, such as the current debate over the issue of comparable pay for jobs of

comparable worth. All of these uses have underscored more than ever before the importance of obtaining a complete and accurate count of the population as well as

accurate data about characteristics.

Yet to obtain highly accurate data costs money. The 1980 census cost close to

Sl.l billion dollars-about $4.75 for each inhabitant of the United States (U.S. Bureau

of the Census, 1983. p. 88). The per capita amount is small compared with the per

case cost of most government and private-sector sample surveys. Moreover, this total

cost includes planning, collection, and processing activities that spanned most of a decade and provided data that are of value for the decade and beyond. Nonetheless, census costs exceeding $1 billion excite comment and invite close scrutiny to determine

how they might be reduced. Recently in Canada, the quinquennial census scheduled for

1986 was cancelled because of budget constraints facing the government; it was subsequently reinstated, however, in response to widespread public expressions of concern

and its demonstrated cost-effectiveness compared with alternatives. The U.S. decennial

census is constitutionally mandated; nevertheless, pressures are likely to be severe in

the coming years to attempt drastic cost reductions both in census planning activities

and in the enumeration, despite the fact that, compared with other ways of obtaining

comparable information, the census is still cost-effective.

The Census Bureau's own research has shown that there were inaccuracies in the

1980 census, both of undcrenumcration (that is, persons who were missed) and overenumeration (that is, persons who were inadvertently counted twice or otherwise included

when they should not have been). Evaluation studies generally point to the conclusion

that the 1980 census produced a small net undercount of the total population-that is,

the census count, including erroneous enumerations, fell somewhat short compared with

an independent demographic estimate. Most significantly, important race, sex, and age

subgroups of the population experienced differential rates of net undercount. There is
51

strong evidence that the black population experienced a net undercount of about 5

percent nationwide. Black men ages 25-54 appear to have had the highest net

undercount rates-close to 15 percent on average (Passel and Robinson, 1984, Table 3).

Coverage estimates for whites and other races are difficult to derive because of the

lack of reliable estimates of net legal and illegal immigration. Making a range of

reasonable assumptions about the size of the illegal alien population, it appears very

likely that whites and other races experienced net undercount in the 1980 census, but

that the rate or undercount was smaller, perhaps significantly smaller than the 1.5 percent rate estimated for 1970 (see Passel et al., 1982. pp. 6-8).

Differential net undercount means possible inequities in redistricting, fund allocation, and provision of social services based on census data as well as possibly erroneous conclusions drawn from studies used as the basis for antidiscrimination policies and

lawsuits and other socially important purposes. The belief that errors in the census

affected representation and fund allocation gave rise to an unprecedented number of

lawsuits following the 1980 census. By October 1981, over 50 suits had been filed

challenging the census results (U.S. Bureau of the Census, 1983, p. 85). Currently, the

judge assigned to a major case in which the State and the City of New York arc suing

to have the Census Bureau adjust the 1980 census counts is reviewing testimony and

preparing to hand down a decision; 23 other cases are awaiting settlement of the New

York suit. Analyses by Kadanc (1984) and Gilford (1983) indicate that the

apportionment of congressional scats may have been affected by the differential

undercount. For example, Kadanc found that if one of the sets of estimates produced

from the 1980 Post-Enumeration Program evaluation were used to adjust the census

results, California would have received an additional seat at the expense of

Pennsylvania.

5. Proposed Changes in Methodology

Not surprisingly, many ideas have been proposed by the Census Bureau and others to

improve the decennial census. Some arc directed principally at improving coverage and

reducing differential coverage errors. One idea in this class is to match administrative

records, such as driver's license lists and other sources, against the census on a scale

even larger than that used in 1980 to identify people who should be followed up to

determine if they were improperly omitted from the census count. Other ideas are

directed principally at reducing costs. One such approach is to make use of sampling,

not only to obtain information on characteristics, as is currently standard decennial

census practice, but also as part of the procedure to obtain the count. For example,

one could attempt contact with a sample of households that do not mail back their

questionnaires, rather than all nonrespondents. in the follow-up stage of census operations. Special coverage improvement procedures could also be carried out on a sample

basis.

Two important themes stand out in current discussions of methodology for the decennial census. One relates to the degree of emphasis that should be given to counting versus estimation. A census, no matter how diligently administered, can never be

complete or without error. Moreover, in current census methodology, not every record

corresponds to a person actually named on a questionnaire; for example, some records

(about 1 percent in recent censuses) represent imputations in situations in which there

is good evidence that a housing unit is occupied but repeated efforts have failed to

find the residents. Hence, a census, strictly speaking, provides an estimate of the

population.

From this recognition has come a view of the decennial process that emphasizes

estimation and argues that some of the resources for conducting the census should be

shifted from traditional coverage improvement procedures to developing the best possible estimates of the total population and subgroups. Input to the decennial year
52

population estimates, in one version of this view (Ericksen and Kadane, 1985), would

include not only a well-conducted census, but also information obtained from various

programs conducted on a sample basis that would provide a basis for adjusting the

census field counts. (Such programs might include matching of samples of

administrative lists to census records and follow-up of a sample of households that did

not respond to an initial follow-up effort.) Whatever the degree of emphasis placed on

estimation, the known errors and the incompleteness of the census count mean that the

issue of adjusting census figures needs to be addressed.

The other theme relates to the critical importance of evaluation programs in the

methodology of the decennial census. Politicians, policy analysts, statisticians, economists, demographers, other social scientists, and users of census data in all sectors

have expressed divergent views regarding the most appropriate methodology for conducting the census. But however they view the census, there is substantial agreement

on the importance of evaluating the completeness and accuracy of census statistics.

The Census Bureau has conducted formal evaluation programs for every census

since 1950 (U.S. Bureau of the Census, no date). All of the techniques used to date in

this country and abroad, including demographic analysis, reverse record checks, administrative record matches, and post-enumeration surveys (whether recanvassing selected

areas or matching independent surveys to census records), have important flaws. In

the United States today, the absence of adequate data for estimating net immigration,

whether of legal or illegal residents (Marks, 1980), poses particularly severe problems

for evaluating the census count even at the national level using the demographic

method. Furthermore, if evaluation results were to be used for census adjustment

purposes, then reasonably accurate information on the errors of evaluation estimates

would also be needed. Nevertheless, with concern over possible inequities in political

representation and the distribution of large amounts of federal dollars as well as concern over the adequacy of the data for analysis of the socioeconomic status of important population groups, there has never been a greater need for thorough evaluation of

the decennial census. This evaluation is necessary whether the object is to inform

users of known errors in the census or actually to adjust census results.

While there is widespread agreement that evaluation is important and that the issue

of adjustment must be faced, many decisions on methodology for 1990 remain to be

made. It is clear that there is no lack of ideas and suggestions that appear useful to

investigate. It is also clear that the process of determining a reasonable methodology

for 1990 will involve difficult choices. The Census Bureau has stated (Bailar, 1984, p.

259) that its minimum goals for 1990 are to:

la) Conduct the 1990 Census without increasing the per-housing-unil

cost in 1980 dollars, (h) Expedite the availability of the data to

users, (cl Maintain a high rate of overall coverage and improve the

accuracy of small area data while reducing the overall differential for

population groups and geographic areas.

It may be possible to design a methodology that makes gains in the desired direction

on each of these dimensions. The more likely situation is that it will be possible to

make progress on one or two dimensions but at the price of giving up improvements on

the others. Explicit trade-offs reflecting costs and benefits will need to be made in

the choice of methodology for 1990 and beyond (see Keyfitz, 1979). Because the high

costs of censuses and the compressed time frame within which they are carried out

make mid-course corrections impossible, it is essential that the methodology to be used

be thoroughly tested.
53

6. Independent Reviews or Decennial Census Plans

The Census Bureau is actively working on methodology for the 1990 census and has

assembled staffs to plan the census and specifically to work on issues of undercount

and the possible adjustment of census counts. For many decades, the Census Bureau

has also actively sought outside independent review of its plans and proposed procedures. In addition to ongoing advisory committees involving various professional disciplines and advisory committees representing the interests of population groups for

whom census results are particularly important, the Census Bureau has asked the

National Research Council (NRC) and the American Statistical Association (ASA) to

conduct special studies of the decennial census. The report of the Panel on Decennial

Census Methodology represents the fourth outside review conducted in recent years of

key aspects of modern census methodology. A brief discussion of the scope and thrust

of the predecessor NRC and ASA studies can help place this latest study in context.

6.1 The 1969-1972 NRC Advisory Committee on Problems of Census Enumeration

The Census Bureau sponsored a study in 1969 by a committee of the National Research

Council to provide advice on ways to improve completeness of coverage in the decennial census and intcrccnsal household surveys. (The U.S. Office of Economic Opportunity and the Manpower Administration of the U.S. Department of Labor also contributed support for the study.) The Advisory Committee on Problems of Census

Enumeration issued its final report, America's Uncounted People, in 1972. The report

focused on the need to understand the social and psychological context in which

undercount occurs. For example, the committee noted that people may be missed in

central city areas because, although members of extended families, they are not

attached to a family or household residence, which is the basic unit of enumeration in

the census and household surveys. The committee strongly recommended that the Census Bureau broaden its research strategy and knowledge base to include methods and

concepts not typically embraced in survey research. The report included specific

recommendations to conduct experimental studies of questionnaire wordings and formats

and their effects on respondents; explore the utility of communication research for

better understanding the reasons for census and survey undercoverage; and carry out

localized participant-observer studies to learn more about the impediments to census

data collection in different kinds of areas.

6.2 The 1978 NRC Panel on Decennial Census Plans

The Census Bureau asked the National Research Council again in 1978 to review decennial census methodology, specifically the plans for the upcoming 1980 census. The

NRC's Committee on National Statistics set up the Panel on Decennial Census Plans,

which worked within a very short time span, to: (I) examine field procedures, questionnaire design, and special procedures designed to improve the 1980 census coverage,

(2) review proposed procedures for handling contested counts, (3) investigate the feasibility of adjusting census counts, and (4) consider evaluation plans for the 1980

census. The panel's report. Counting the People in 1980: An Appraisal of Census

Plans, made recommendations in many areas. This panel repeated the call of the

earlier committee for imaginative work on the cultural and social problems associated

with census-taking. In the area of adjustment, the 1978 panel concluded (National Research Council, 1978, pp. 132-133) that: "methods of adjustment with tolerable accuracy are feasible" and "on balance an improvement in equity would be achieved." The

panel supported implementation of procedures to adjust population counts for

underenumeration for purposes of fund distribution and expressed confidence in the

Census Bureau to determine the best technical procedures for adjustment. The panel
54

recommended thit adjustment "not be applied to the counts used for legislative apportionment nor to the body of census data on the characteristics of the population."

6.3 The 1981-1982 ASA Technical Panel on the Census Undercount

The Census Bureau asked the American Statistical Association in 1981 to convene an

expert group to review the methods and results of the programs used to evaluate completeness of coverage in the 1980 census and to make recommendations regarding research in the areas of coverage evaluation and adjustment of census counts. This

panel made a number of specific research suggestions and also recommended (ASA.

1984, p. 256): "that the Bureau of the Census sponsor an outside technical advisory

group on undercount estimation and related problems."

6.4 The 1984 Panel on Decennial Census Methodology

In response to the recommendation of the 1981 ASA panel, the Census Bureau asked

the Committee on National Statistics at the National Research Council to establish the

Panel on Decennial Census Methodology. The charge to the panel was for an

Investigation of three major issues from a technical viewpoint, setting aside legal considerations:

(1) Adjustment of census counts and characteristics, including exploration

of formal criteria to evaluate measures of undercount and alternative

adjustment procedures;

(2) Uses of sampling in the decennial census, including investigation of

whether, for a given cost, the sampling of lists and areas to improve

coverage and sampling of nonrcspondents for follow-up can improve

accuracy for the total population and for important subgroups;

(3) Uses of administrative records, including investigation of various types

of records to determine their possible utility in improving the accuracy

of census counts and the efficiency of census operations.

The panel interpreted this charge to include investigation of closely related topics,

notably methods of coverage evaluation and improvement. Coverage evaluation programs provide the necessary input data for any adjustment and serve the important

function of apprising users of the quality of the census counts. Procedures for

coverage improvement were viewed by the panel as necessary and desirable whether or

not an adjustment procedure is incorporated into census methodology. The panel also

investigated uses of census data and their dependence on the accuracy of the census

figures. Proper evaluation of the consequences of changes in collection methodology

requires an understanding of important uses of the data being collected.

The charge to the panel related to analysis of decennial census methodology and

not to other population programs of the Census Bureau. However, during the panel's

work, it was clear that the census could not be considered completely in isolation.

Demographic and related social and economic statistics are used continually over the

decade following each census, and current information is needed for these uses. The

Census Bureau has a number of formal programs for updating some of the census information. Hence, the census is the central part of a broader statistical system designed

to produce data needed to implement legislation, assist in decision making both by industry and government, and help understand changes taking place in our society. Although the panel did not undertake a study of population statistics programs other

than the census, the panel considered the quality of census data compared with the
quality of postcensal population estimate*. The panel recommended (hat the Centui

Bureau assess the need for a mid-decade census in 1995 in light of the impact! of errors in postcensal population estimates on major data uses, such as fund allocation.

The work of the panel differed in several important ways from the efforts of ill

predecessors. This was the first panel asked explicitly to consider important changes

in decennial census methodology from the perspective of cost as well as effectiveness.

A theme running through the charge to the panel is to design a methodology that improves accuracy compared with previous censuses but costs no more, and ideally less,

in constant dollars.

Other important differences have to do with the timing of the panel's work in

relation to the cycle of decennial census planning. The panel was convened at a point

in the cycle when it could benefit from the availability of extensive material regarding

the experience in the most recent census. At the same lime, the panel carried out its

work in an early stage of the planning cycle for the next decennial census before decisions on methodology were fixed. Hence, the panel was in an unusually good position

to provide suggestions and guidance regarding the research and testing program for

1990.

7. Major Themes of the Panel's Report

Several themes run through the report of the panel, The Bicentennial Census: New

Directions /or Methodology in 1990. The first major theme can be expressed as the

need for balance between traditional and new procedures in the choice of census

methodology for 1990. Indeed, balance has characterized the historical evolution of

decennial census methodology. The report does not propose that the Census Bureau

make radical innovations in decennial census methodology in the near term. The census is a massive and complex operation, and major changes should be made only with

care and after thorough evaluation. Nonetheless, the report expresses the belief that

it is important to implement changes on some dimensions for 1990 and to undertake

planning that may lead to further changes in the future.

Most important, the report argues for balance between efforts to achieve a complete enumeration and efforts to improve the accuracy of census figures through

adjustment procedures. The panel believes that adjustment cannot be viewed as an

alternative to obtaining as complete a count as possible through cost-effective means.

The United States has a long tradition of a census as a complete enumeration in which

it is a civic responsibility to participate in the census process. It is important to

continue this tradition and important that census methodology should strive for a complete enumeration via counting procedures, including the use of cost-effective special

coverage improvement programs. However, the report also states that the ultimate goal

of the census should be the accuracy of the census figures. The evidence is overwhelming that no counting process, however diligent, will in fact enumerate everyone.

Hence, the report recommends that the Census Bureau carry out a vigorous program of

research on coverage evaluation and adjustment methods that, if successful, would

permit adjustment of census figures as part of the methodology for the 1990 ceniui.

A second and related theme concerns cost-effectiveness. The panel did not

attempt to apply formal cost-benefit analysis to decennial census methodology, but endeavored to identify those proposed changes that show the most promise of improving

accuracy without increasing costs or of reducing costs without importantly impairing

accuracy. In this regard, the panel's recommendation for research designed to develop

appropriate and feasible methods of adjustment of the census counts, together with the

Census Bureau's stated goal to contain costs for the 1990 census, implies that some

budget resources must be shifted from coverage improvement to coverage evaluation

and adjustment. Specifically, the panel argued in its report that coverage improvement

programs used in previous censuses should be carefully reviewed to determine their
56

efficacy. Costly programs that neither correctly added significant numbers of people to

the count nor importantly reduced differential undercount should be dropped from the

Census Bureau's plans for 1990. Effective programs should be further refined through

testing and research, and the budget should make room for testing some new ideas in

this area.

While not favoring extensive use of sampling to obtain the count, the panel in its

report supported research on sampling in the later follow-up stages of census operations and in some coverage improvement programs, such as the program to recheck the

vacancy status of housing units. Limited use of sampling may effect measurable cost

savings with minimal sacrifice of accuracy. Careful use of sampling for certain

coverage improvement programs may. in fact, improve accuracy by reducing duplications

and other erroneous enumerations, in addition to identifying missed households and

people.

In considering cost and accuracy, the panel stated its belief that it is important to

look at the characteristics data collected in the census as well as the population

count. There is strong evidence that important subject items have severe reporting

problems. The panel recommended a strategy of looking closely at each item proposed

for inclusion on the questionnaire to determine: (I) the need for that item, (2) the

level of geographic detail required by users, and hence whether the item must be asked

of all households on the short form questionnaire, or whether it can be asked of a

sample on either the long form or on a much smaller follow-on survey, and (3) whether

some other source could provide higher-quality data. The panel suggested exploring

the use of administrative records together with sampling to obtain data on some housing structure characteristics. Such data could be more accurate than individual

responses on the census form. Costs initially may be high, but should decline over

time. This particular use of administrative records has the advantage that it should

present no actual or perceived threat to individual privacy.

A third major theme of the report concerns the strategy for designing the 1990

census, whatever the particulars of the methodology may turn out to be. The research

plans drafted by the Census Bureau staff are extremely comprehensive and ambitious.

The staff has clearly tried to include all reasonable ideas for consideration in the research and testing program. The panel commended the Census Bureau's efforts to

design and carry out a thorough research and testing program that will support sound

decisions regarding methodology for the 1990 and later censuses.

The panel expressed its belief, however, that in most areas the Census Bureau must

choose among all the ideas and procedures proposed for testing, given constraints on

available staff and budget resources and the limited time available to analyze test results and use them to guide decisions on methodology. The exception concerns

research related to adjustment, including research on coverage evaluation methods. In

this area, the panel stated that research must proceed on a broad front if effective

methodologies are to be developed for 1990. In other areas, the panel endeavored to

recommend strategies for choosing priority projects for inclusion in the 1990 census

research and testing program and also recommended the use of less costly research

methods, where appropriate, including more detailed analysis of 1980 census results, in

place of full-scale field tests.

8. Overview of the Panel's Recommendations

In the remainder of the paper, we summarize the recommendations of the Panel on

Decennial Census Methodology.
8.1 Recommendation! on Adjustment of Population Counts

The first issue in the charge to the panel was that of adjustment of the census counts.

Based on review of the evidence regarding coverage and other kinds of errors in the

census and of the literature on the important uses of census data, the panel found a

need for adjustment to improve the accuracy of the census numbers, particularly to

reduce differential coverage errors across geographic locations and demographic

groups. The panel was led to recommend development of adjustment procedures, but as

a complement to-not a substitute for--continued efforts to improve census coverage.

If public perception of the importance of being counted should deteriorate, this would

have serious consequences for the accuracy of the figures, adjusted or not.

Recommendation. Completeness of the count is an important goal, both

for ensuring the accuracy of the census and for establishing the credibility

of the census figures among all users. Adjustment should not be viewed

as an alternative to obtaining as complete a count as possible through

cost-effective means. Severtheless. the ultimate goal is that of the accuracy of the published figures. Given the likelihood that the census will

continue to produce different rales of undercoverage for various population

groups, and given the equity problems caused thereby, we recommend that

work proceed on the development of adjustment procedures and that adjustment be implemented if there is reasonable confidence that it will

reduce differential coverage errors.

The panel also investigated criteria for evaluating the numbers produced by the

census (based on cither unadjusted or adjusted counts), considering both the errors in

the numbers themselves and the resulting loss to society due to erroneous treatment of

political jurisdictions in terms of representation, fund allocation, and other uses of the

data. The panel considered various yardsticks or loss functions, that is, numeric measures of the impact of census errors, from the viewpoint of the data user and as they

relate to adjustment. The discussion of this topic in the panel's report notes that no

adjustment procedure can be expected to simultaneously reduce the error of every

piece of census information for every geographic area; rather, there is an important

net social gain if differential coverage error is generally reduced. The panel expressed

the belief that it is more important to reduce the overall error per person than the

overall error per place and recommended that loss functions for measuring total error

take into account the population size of each jurisdiction. In discussing technical

considerations concerning choice of loss functions, the panel concluded that good

adjustment procedures should be expected to perform well for a range of loss

functions. Moreover, no type of jurisdiction should have substantial reason to believe

that its population could have been estimated more accurately some other way Where

the choice of adjustment procedure depends importantly on the choice of loss function,

this suggests that the particular adjustment procedure has weaknesses that need to be

addressed.

Recommendation. In measuring the total loss associated with an

adjustment procedure, we recommend that the contribution to this loss

attributable to a geographic region should reflect its population silt.

Thus, we recommend against loss functions based solely on the number of

political entities losing or gaining through adjustment.

Recommendation. We believe that, in general, the results of an adjustment

are likely to be affected more by the quality of coverage evaluation data

and the models and methodology used than by the choice of loss
58

functions. Given a family of loss functions with relatively similar

objectives, it should be possible, and desirable, to determine an adjustment

procedure that has good performance for most or all of them. We

recommend that the Census Bureau investigate the construction of

adjustment procedures that are robust to a reasonable range of loss functions.

The panel considered the problem of estimating the likely range of error introduced by the particular procedure adopted for an adjustment. Although error can be

measured only imperfectly, information about the distribution of error is important in

the same way that sampling variances for sample surveys provide useful information.

Recommendation. We recommend that the Census Bureau explore methods

for providing estimates of errors associated with estimates of census overand undercoverage. with a view to publishing such error estimates along

with coverage evaluation results and any adjusted census data that may be

issued.

Adjustment of census data could create problems of inconsistency between aggregate statistics and microdata from the census. The panel stated its belief that internal

consistency is an important quality for general purpose statistics, such as those produced by the decennial census, which have a wide range of output and many uses. The

report discusses reasons to carry down any adjustment of population estimates for

larger geographic areas to the level of the individual micro-records and reviews

methods, such as weighting and imputation, for accomplishing this.

Recommendation. The panel believes that it is important to strive for

internal consistency of published census figures. Should adjustment appear

feasible and effective, methods exist for distributing adjusted totals for

aggregated groups down to subgroup values. We recommend that one of

these methods be used to achieve internal consistency of census figures.

Adjustment also presents problems of timing. Current law requires submission of

state population counts within 9 months after Census Day for purposes of reapportionment and of small-area counts within 12 months after Census Day for purposes of redistricting. The report discusses the pros and cons of various scenarios with regard to

release of adjusted data if it proves impossible to implement a full-scale adjustment in

time to satisfy the above constraints. Congress clearly will need to stipulate which

scenario is preferable for apportionment purposes.

Recommendation. Census data used for reapportionment and redistrieting

are required by law to be produced no later than specific dates, ft is

possible that adjustment of the 1990 census will prove feasible and

effective in all respects, except for the ability to meet the required

deadlines. This should not necessarily preclude subsequent issuance of

adjusted data for other uses. In this situation, we recommend that the

Census Bureau seek determination by Congress of whether it desires that

adjusted data be used and will therefore extend the deadlines, or wishes to

adhere to current deadlines and will therefore stipulate the use of

unadjusted (or partially adjusted) data for reapportionment and

redistrieting.
The panel reviewed possible technical approaches to the use of data from coverage

evaluation programs for adjusting the raw census figures. The review covered procedures Tor starting out, that is, for developing estimates for a limited number of large

geographic areas, and procedures for carrying down, that is, for using the large-area

estimates to develop adjustments for small areas and ultimately for the microdata

records. The discussion of this topic in the report considers the Census Bureau's plans

for research and testing of adjustment procedures in upcoming pretests and makes

recommendations for priority research areas.

Recommendation. The panel recognizes that considerable work is still

necessary and likely to lead to improved procedures lor adjusting census

data. We therefore support the Census Bureau's stated plans to pursue,

internally, research and development of adjustment procedures, and we also

recommend that the Census Bureau vigorously promote and support related

statistical research in the academic community.

Recommendation. The panel supports the Census Bureau in its plans for a

1986 pretest of adjustment operations, including the production of mock

tabulations of adjusted census data. We recommend analysis of the

resulting adjusted and unadjusted data sets, to help identify the strengths

and weaknesses of the particular methods tried.

Recommendation. We recommend that research on adjustment include: (I)

investigations of the assumptions underlying the procedures. (2) an attempt

to evaluate empirically the more important of the assumptions as well as

the sensitivity of methods to violation of assumptions. (}) study of

methods used for carrying down estimates to lower levels of aggregation,

and (4) a study of the impact of adjustment on uses of census data.

8.2 Recommendations on Methods to Measure the Completeness of Census Coverage

For adjustment to be feasible, evaluation programs must be good enough to provide

estimates of net undercoverage that are reliable for at least large geographic areas and

have error properties that are broadly understood. Coverage evaluation programs also

provide valuable information for users of the data and for the Census Bureau in planning subsequent censuses. Although in general the panel recommended that the Census

Bureau narrow its 1990 census research and testing objectives, in the area of coverage

evaluation the panel expressed the belief that it is too soon to focus on one method to

the exclusion of others.

The report reviewed the problems associated with each of the major methods of

coverage evaluation and the Census Bureau's current plans for research and testing

directed toward coverage evaluation of the 1990 census. The panel argued against the

Census Bureau's decision to concentrate on post-enumeration (or possibly pre-enumeration) survey methodology as the principal means of coverage evaluation in 1990, noted

that the Census Bureau should not put itself in the position of lacking a means of adjustment if there are problems with the operation for matching survey with census

records, and urged completion of 1980-based studies related to coverage evaluation.

Recommendation. We recommend that the Census Bureau conduct

research and tests of alternative coverage evaluation methodologies in

addition to the post-enumeration survey, specifically reverse record

checks and systematic observation.
60

Recommendation. We agree that matching algorithms are very important to

the success of several adjustment methods. We recommend that the

Census Bureau investigate the development of a fallback position in case

adequate matching is not available in 1990.

Recommendation. We recommend that the Census Bureau complete and

report analyses of 1980-based tests related to coverage evaluation,

especially the Census/CPS/IRS Match Study.

The panel considered possible improvements and recommended priority research

areas for each major coverage evaluation method in turn. The demographic analysis

method, which uses data from independent sources including birth and death records to

estimate the number of persons at the time of the census in a given age-race-sex category, currently suffers from the absence of data on illegal aliens. The panel

recommended research into using demographic analysis for estimates of the native-born

population. The reverse record check method, which traces the current location of a

representative sample of newborns, immigrants, and persons counted in the previous

census or coverage evaluation program, has been widely used in Canada. Tracing is

more difficult in the United States because of the 10-year interval between censuses as

opposed to 5 years in Canada. The panel recommended completion of a current experiment to test alternative methods of tracing. The report discusses extensively the

method of post-enumeration (or pre-enumcration) surveys, in which a sample of

households is interviewed and matched with records in the census, and identifies several problem areas for particular attention in the Census Bureau's research.

Finally, the report discusses the idea of using some kind of systematic observation

procedure whereby persons residing in a sample of areas would provide independent

population estimates. The sample should include but not be limited to areas that have

proved particularly hard to count in previous censuses This method might surmount

the problem observed repeatedly in the history of coverage evaluation, namely that

persons who are missed by the census are also likely to be missed by an independent

survey or other data source.

Recommendation. We recommend that the Census Bureau conduct research

into using demographic analysis to develop estimates of coverage for the

native-born population. The research should consider whether these

estimates could usefully be combined with other estimates of coverage.

Recommendation. We recommend that the Census Bureau move quickly to

complete the Forward Trace Study to determine the feasibility of using

forward trace methods in a reverse record check program for 1990. If the

methodology is effective, a national sample for this purpose needs to be

initiated by 1986.

Recommendation. We support the Census Bureau's research directed toward

developing the 1990 Post-Enumeration Program and recommend that such

research emphasize the following areas:

fa) Reduction of post-enumeration survey nonresponse:

(b) Reduction of unresolved matches between records for

individuals listed in the post-enumeration survey and the

decennial census:
61

fc) Validation of the essmmptums tmd/tr development of

alternative metkodologies •"x* respect to netting-out of

overcounts and undertomnts "ilk reference to tke place o/

enumeration: and

(d) Investigation of alternatives to ike assumption that ike

inclusion of indirtdualt in Ike post-enumeration survey is

unrelated to tkeir inclusion in ike decennial census and ike

estimation of ike strengtk of tkis relation.

Rtcoameadaiioa. We recommend that ike Census Bureau inmate a

research program on systematic observation witk a view toward ike use of

tins method for a sample of areas at the time of ike 1990 census.

In the area of adjustment-related research, including coverage evaluation methods,

the panel acknowledged that many technical and operational issues need to be resolved

if adjustment procedures are to be developed in time Tor their use in the nation's bicentennial census in 1990. Overall, while much effort will be required, the panel expressed optimism that substantial progress can be made.

8.3 Recommendations on Procedures for Improving the Count

The panel considered not only methods for adjusting the census figures, but also procedures for improving the counts obtained in the field. Most programs directed toward

coverage improvement arc expensive. They may also introduce error by duplicating or

otherwise erroneously adding persons. In general, however, the panel determined that

the costs of wcll-dcsigncd and well-executed coverage improvement programs represent

loncy well spent for improving the census figures.

The panel noted the importance of gaining understanding of the problems of undcrcount and overcount in the census, as the evidence indicates that the field enumeration is not equally effective for all population groups.

Recommendation. Wt recommend that Ihe Census Bureau assign a high

priority to the completion of studies of undercount and overcount in the

1980 census.

Recommendation. Il> recommend that the Census Bureau set up a

timetable and assign staff to permit completion of the analysis of 1990

coverage evaluation results in lime to be used in planning the first pretest

of the 2000 census.

The panel considered priorities for research and testing directed toward improvement of items on the questionnaire that relate to coverage, including the questions on

race and Hispanic origin. It is important to understand what responses to the race and

ethnicity questions mean to develop appropriate estimates of coverage rates for race

and Hispanic groups and to relate them to vital statistics.

Recommendation. We recommend thai the Census Bureau test a variety of

question designs for the race and ethnicity information to be collected in

the 1990 census, including some thai combine the collection of information

on Hispanic origin with the other race and ethnicity information.

m

64-670 0-86
62

Recommendation. We recommend that the Census Bureau, in addition to

other methods that it has traditionally employed, use the technique of

focus group discussions as one means to develop questions on particularly

sensitive items such as race and ethnicity.

Recommendation. We recommend that, in 1990 as it did in I9S0. the Census Bureau collect, tabulate, and release data on race and ethnicity in such

a way that the data can be reaggregated as necessary to obtain maximum

feasible comparability with 1980 and 1970.

Recommendation. We recommend that the Census Bureau, the National

Center for Health Statistics, and other relevant federal agencies work

closely together to design questions and response editing rules on race and

ethnicity that minimize conceptual differences between census and vital

statistics records to the extent feasible. The Office of Management and

Budget should act as necessary to facilitate such coordination.

The panel evaluated experience in the 1970 and 1980 censuses with questions on

the short form designed to aid in achieving a complete and accurate count, such as

questions probing for a complete roster of household members. Increasing problems are

posed for an accurate count by the mobility of the population and recent trends in

living arrangements that have resulted in growing numbers of persons with two or

more usual residences (for example, retired people with summer and winter homes).

The panel suggested a question for testing directed toward improving coverage of

young adults and children in hard-to-count areas.

Recommendation. Iff recommend that the Census Bureau give high priority

in its planning for 1990 to research and testing of questions and enumeralion procedures that address problems of accurately counting persons in

the process of moving, households with second 'vacation} homes, and persons with more than one usual place of residence.

Recommendation. We recommend, as one procedure to consider for

improving coverage of hard-to-count groups, that the Census Bureau

pretest a question asking parents for names and addresses of children who

are not part of the household. This question should be included in the

1986 pretests.

The panel also made an overall assessment of special enumeration procedures

designed to improve the count. While believing that programs such as the rcchcck of

vacant units can make important contributions to improving coverage, the panel did not

subscribe to the view that every coverage improvement idea that is suggested or has

been used in the past should be included in the plans for the next census. The panel

recommended paring down the list of programs to be considered for 1990 and the list

requiring early field testing.

Recommendation. We recommend that the Census Bureau review coverage

improvement programs used in past censuses and proceed with research and

testing directed toward use in 1990 of those programs that: II) exhibited

a high yield in terms of numbers of missed persons correctly added to the

count and/or contributed significantly to reducing differential

undercoverage. (21 exhibited low-to-moderate costs per person correctly

added, and (3) did not add many persons incorrectly. Programs that do

not satisfy these criteria should be dropped from consideration unless: (I)
63

the program exhibited low total dollar costs and had demonstrable public

relations or goodwill value in previous censuses or (2) there is some

particular reason to believe a revised program will yield greatly improved

results.

Recommendation. We recommend that the Census Bureau conduct full-scale

pretests in 1986 only of those coverage improvement programs that require

such testing. Furthermore, we recommend that the Census Bureau use

focus groups that include members of hard-to-count populations as one

means to explore coverage improvement techniques and to narrow the

range of options to be field-tested.

8.4 Recommendations on Uses of Sampling and Administrative Records

The panel reviewed two major methods that have been proposed to improve the

cost-effectiveness of the decennial census•the use of sampling in obtaining the count

and the use of administrative records. With regard to sampling for the count, the

panel noted problems of replacing the census with a large sample survey: sampling on

the scale necessary for satisfaction of present demands for small-area data would complicate field operations, reduce costs relatively little, and probably exacerbate problems

of coverage errors compared with a census. The use of sampling for follow-up of

households that do not return their census questionnaires has some of the same drawbacks, but sampling could prove cost-effective in the final stages of follow-up in which

it is very expensive to count an additional person. Although the Census Bureau has

dropped plans to study the use of sampling for follow-up and for coverage improvement

programs such as the rcchcck of vacant units in 1986, the panel expressed support for

research in these areas. The panel also supported further testing of telephone

follow-up of nonrcsponding households, which was tried experimentally in 1980.

Finally, the panel underscored the need to maintain machine-readable records of the

follow-up history of individual households that will permit detailed analysis and simulation of different sample designs.

Recommendation. "V recommend that the Census Bureau not pursue

research or testing of a sample survey as a replacement for a complete

enumeration in 1990.

Recommendation. lie recommend that the Census Bureau include the

idling of sampling in follow-up as part of the 1987 pretest program. We

recommend that in its research the Census Bureau emphasize tests of

sampling for the later stages of follow-up.

Recommendation. We recommend that the Census Bureau consider the use

of sampling for those coverage improvement programs that are implemented

in the final stages of census operations and where there is potential for

significant cost savings. We recommend that the Census Bureau simulate

sampling in the Vacant/Delete Check program in an upcoming pretest.

Recommendation. We support the Census Bureau's plans for further testing

of telephone follow-up procedures in 1986. We recommend that the Census

Bureau review the implications for sample-based follow-up operations of

the operational difficulties that were encountered in the 1980 telephone

experiment.
64

Recommendation. We recommend that the Census Bureau keep machinereadable records on the follow-up history of individual households in the

upcoming pretests and for a sample of areas in the 1990 census, so that

information for detailed analysis of the cost and error structures of conducting census follow-up operations on a sample basis will be available.

In addition to evaluating the uses of sampling for obtaining the basic head count,

the panel reviewed the use of sampling for content items in the census. Historically,

in every census since 1940, some items have been asked of only a sample of the population in order to reduce response burden and processing costs while obtaining the

benefits of additional data. Sample designs and sampling fractions have differed in

recent censuses. The Census Bureau considered a design for 1990 that would include a

short form containing items asked on a 100 percent basis, a long form containing additional items asked of a large sample, and a follow-on survey of a small percentage of

short-form households administered within a few months of Census Day that would obtain yet other information. The panel did not offer specific recommendations in this

area, but noted that the criteria for including items in the follow-on survey have not

been explicitly articulated but should be to permit thorough assessment of the need for

the survey and for the inclusion of particular items.

Recommendation. We recommend that the Census Bureau refine and make

more explicit its criteria for inclusion of Hems in the proposed follow-on

survey that is being considered for the 1990 census.

Finally, the panel investigated the use of administrative records for improving the

accuracy of content items. The concern over completeness of population coverage in

the census can obscure equally valid concerns over the accuracy of the content. There

arc well-documented problems with the reporting of content items such as income,

utility costs, and age of structure. The panel recommended research and testing

directed toward improving the data quality of key items. The research program should

include design of operations to verify, and possibly adjust, responses as part of the

census operation and should investigate the possibility of obtaining some items, such as

housing structure information, from administrative records sources.

Recommendation. We recommend that the Census Bureau conduct research

and testing in the area of improved accuracy of responses to content items

in the census. We recommend further that the content improvement procedures examined not be limited to reinterviews of samples of respondents,

but include the use of administrative records.

Recommendation. We recommend that the Census Bureau investigate the

cost and feasibility of alternative ways of obtaining data on housing

structure items. Possibilities include: (I) obtaining housing structure

information on a sample basis from administrative records and using this

information to verify and possibly to adjust responses in the census; (2)

obtaining structure information solely from administrative records and

dropping these items from the census: and (Si asking structure questions of

a knowledgeable respondent such as the owner or resident manager. We

recommend that any trial use of a 'knowledgeable" respondent procedure

include a check of the data obtained from such respondents against data

from administrative records.
tJ R«omm«o<i«Iioo» ea Bewarch aad Testing for th« 199* Ccasu

The panel reviewed the Census Bureau's research and testing plins for the upcoming

1990 census, with particular emphisis on the plans for pretests in spring 19g6. The

panel expressed several major concerns with the 19*6 research and testing program

which appeared too ambitious for the time remaining before the census and for the

staff and budget resources likely to be available, particularly .f key data are to be

analysed in time to support major decisions. In the panel's view the program also

placed too much emphasis on field testing over other kinds of research, including

further analysis of existing data. The panel suggested some ways to scale back the

I9S6 testing program.

Recoaaeadatioa "> recommend. to ensure cost-tffectire field felling end

preservation of adequate resources for analysis, that the Census Bureau

attempt to identify research and testing proposals for 1986 that:

(a) Can be pursued with other research methods and omitted from

the 1986 field test program:

• lb) Can be safely deferred for research or testing until 1987 or

until the dress rehearsals:

(c) Are unlikely to be viable for 1990 but should be incorporated

on an experimental basis into ihe 1990 census as a test for

future censuses: and

(d) Should be omitted entirely from consideration for the 1990

census, based on previous census experience or other surrey

research results.

Rfcomratndilion We recommend that the Census Bureau make full use of

data from the 1980 census and from experiments carried out in 1980 to

help giude planning for 1990. To this end. we recommend that the Census

Bureau assign a high priority to completion of 1980 census methodological

studies, and we encourage further analysis of these data where

appropriate.

Throughout its report, the panel endeavored to identify priority areas for research and

testing to support the choice of methodology for the 1990 and future censuses in the

VS.
66

Mr. GARCIA. I want to backtrack a little bit if I can. You may

have made mention of it, but I just want to make sure because one

of the individuals who is going to testify, a person who obviously I

have to be quite concerned about, is Mr. Carlucci, who handles the

statistical data on how we are going to reapportion the State of

New York for the 1992 election. But I just want to make certain we

understand.

I remember Detroit, and I remember New York, and I remember

a series of cities that sued the Bureau of the Census and sued on a

question of "adjustment." If, in fact, the adjustment is made, when

will that adjustment be made? When will figures and statistics be

available to the States?

Dr. BAILAR. We are trying very hard to meet a schedule that we

have those be the official census figures, so that they would come

out December 31, 1990.

Mr. GARCIA. Of 1990?

Dr. BAILAR. Right.

Mr. GARCIA. OK. Because then we end up with the constitutional

question of the one man, one vote, and we could end up in another

series of litigation which could probably disturb and overturn everything that we have done in 1990 in preparing for the 1992 reapportionment.

As I said before, Mr. Carlucci, who is from the State of New

York, is going to testify, believes that there is a danger that any

adjustment the Census Bureau makes would, in fact, be troubled by

the same problems as the enumeration. He believes that adjusting

the census will not help groups who are otherwise missed and that,

that it is going to be unfair.

I guess the question is can you tell us do you think this is a problem? When I say you, I am talking about Census Bureau. And if so,

what are you going to do to investigate it or, hopefully, correct it?

Dr. BAILAR. Well, I recognize Mr. Carlucci's concerns, and it is

probably the case that we are not going to be able to make a perfect adjustment, just as we don't have a perfect census. However,

the idea that we will not be improving the census by being able to

adjust for some of the missed people, I think we are going to be

able to do that. And by using the two methods together, by using

both the postenumeration survey and the demographic analysis together, I think we will be able to make real improvements in those

groups of people that we have missed.

Mr. GARCIA. Again, here on Capitol Hill there are those who

might think that an adjustment is a way of influencing the outcome of the census. If, in fact, you decide to adjust the census, how

will you be able to do it without giving interested groups the idea

that the adjustment would be the occasion for a political compromise•and I underline the words "political compromise"•that

would affect the outcome? I guess the bottomline is, How do you

preserve the integrity of the census?

Dr. BAILAR. Well, I think first of all people have to realize that

the census process itself is not a fixed process over the years; that

there have been changes in the censustaking procedures, that we

have made adjustments in the census; that by substituting people

in, by imputing people, we have already made adjustments. That

this final process of adjustment is just at the end•at the very end
67

of a census process designed to count every person that we possibly

can by enumeration methods•then this process would be at the

end to try to correct for the final problems if they still exist.

I think there are some people who will never be convinced that

this isn't a political process. But by the decision process we are

going through, by trying to involve a lot of people outside the

Census Bureau as well as within, we are certainly trying to make

it an unpolitical process.

Mr. GARCIA. Dr. Bailar, again, there are some people who probably say: Well, look, if you are going to adjust the census, the heck

with it. I am not going to participate. You will just count me in

anyway. And others, especially here on Capitol Hill in light of

Gramm-Rudman might even say to you: Just adjust the whole

damn country. Do the best you can and look at the billions of dollars we will save.

What is your response to that?

Dr. BAILAR. First of all, you can't do an adjustment without

taking a census first. An adjustment is based on both the numbers

from a census as well as what you do afterward in a census evaluation or demographic analysis.

We have also looked at the results both in Australia and in England where they have adjusted the census and asked them what

was the public reaction. It was a very "ho-hum" attitude. Most

people just don't realize that you have done it. And it really isn't

different from other things that we do in current surveys. I think

that some people will always be convinced that a group of people

won't cooperate because you are doing something. I don't think

there is any evidence to support that.

Mr. GARCIA. I am not sure if I understand the procedure in the

United Kingdom in terms of taking the dollars, Federal dollars,

and how those dollars are distributed throughout the country. Are

they done by census tracts, or the Government just says we are

going to put so much in Liverpool and so much in Birmingham and

so much in London? How does that work? Because here in the

United States the Coleman Youngs and the Ed Kochs and other

mayors, are concerned because of our funding formula. I am not

sure how the United Kingdom works, as I said before. So it may be

"ho-hum" over there, but the editorial boards and the writers

within the various regions of our country will not take that same

attitude if they feel that that section of the country is not going to

share in what they consider to be a fair procedure. And most of our

moneys are handed out based on population and we have different

levels, which means different funding levels. So this is something

that I feel that they can say ho-hum over there, but I don't think

that will happen here.

Dr. BAILAR. Well, I don't think mayors of cities are going to go

ho-hum about what we do in the census. I agree with you there. In

England, when they adjusted, they made one adjustment and that

was to the city of London. And since that is the biggest city there it

certainly took care of some interests. I am not fully aware of how

they distribute money there.

Mr. GARCIA. Well, because I just feel that we have a different situation here in the United States and everybody is cognizant of

what you are doing. Maybe the average citizen is not, but I can tell
68

you that those of us who are responsible in one form or another for

funding localities, we are quite concerned about the census.

And the second part of it is those politicians, whether at the

State, local, or the Federal level, everyone of us are concerned as to

how those numbers are going to fall as well because that is going

to determine the next 10 years of government.

Dr. BAILAR. Yes, I agree. I think the census counts are very important, and I also think that any improvements that we can make

that we are fully satisfied are going to improve those counts should

be things that people would be satisfied with, if they are fully cognizant of the fact that we are trying to make improvements as we

go along.

Mr. GARCIA. Suppose you come up with a plan that will produce

adjusted numbers and you don't release them because you feel they

don't have to be released.

Dr. BAILAR. Yes.

Mr. GARCIA. What would be the perception in this country of the

Bureau of the Census having two sets of figures?

Dr. BAILAR. Well, the Bureau of the Census would only have one

set of official numbers, and those would be those that we release.

We would, of course, as you have read in the testimony, have these

other unadjusted figures inhouse and we would release them, as we

do every time after a census, in one of our evaluation reports. We

would view those then as a measure to be used for evaluation but

not for adjustment. But they would be released.

Mr. GARCIA. Dr. Fienberg, I guess you have spent enough time

listening to Dr. Bailar. What is your reaction to some of the testimony that she has put forth?

Mr. FIENBERG. Well, I am actually very pleased. As I wrote my

testimony for the record, I was very concerned that a process that

began almost 5 years ago, just after the 1980 Census, working

toward the development of methodology and agreement within the

professional community, had somehow been sidetracked and that

the work within the Bureau had not been progressing at the rate

that I felt was necessary in order to get things done for 1990. As

you mentioned in your opening statement, some people think it is

awfully early to be talking about 1990. My reaction is it may be too

late. And I was very concerned, as I mentioned at the previous

hearing in May, that adjustment had disappeared off of the agenda

of the Bureau. I think Dr. Bailar's statement suggests that the contrary is true, and I think the kind of schedule she proposes is one

that is very compatible with discussions we had at the beginning of

plans for the Committee on National Statistics' panel. I think not

only am I pleased, but if the rest of our panel were to have an opportunity to read this testimony, they would feel that it was quite

responsive to the panel's recommendations of a year ago.

Mr. GARCIA. IS the national academy going to be prepared to play

the role of reviewing and evaluating and making recommendations?

Mr. FIENBERG. I think the academy is very anxious to play that

role. The Committee on National Statistics views its Decennial

Census Panel as its most important panel in the sense that this is

the big statistical activity in this country, and the fact that we can

provide advice and provide input to the Bureau in its decisionmak-
69

ing is very important to the members of the committee. We are

very anxious to be of assistance.

Mr. GARCIA. I guess it takes a while for the academy to prepare

its reports. Do you think that your panel would be able to act in

time to provide meaningful input to the Bureau in a way that is

going to help us politicians in our reapportionment?

Mr. FIENBERG. The academy ha<> its own slow ways, and I fight it

as much as the people on the outside do when I am on the inside.

But we have a variety of other mechanisms for responding; in particular, the mechanism of letter reports, which are brief reports

without the extensive academic-type documentation, which I think

is the kind of response that the Bureau is looking for in the crucial

decisions that it is going to face. I don't see any problem whatsoever in our providing that kind of timely advice.

Mr. GABCIA. Counsel would like to ask a question.

Ms. FERNANDEZ. In your testimony you state that the adjustments will be based on statistical models rather than direct estimates for a block. We have a constitutional requirement that every

10 years we will apportion according to the population in existence.

I would think that your statistical models methods potentially

could lead to more than the 50 lawsuits that the Bureau experienced subsequent to the 1980 Decennial on the question of what is

the actual number of people in a particular district.

How will these adjustment methods have an impact in terms of

those numbers and people's perceptions, especially political perceptions, on whether or not each congressional district is equal to each

other? What do you anticipate in terms of those issues being

raised?

And also, the other question I have is that in your testimony you

also state there is a process where you have outsiders' evaluations,

and an internal technical group within the Bureau of the Census

that will be making a recommendation. But essentially the line in

your testimony that I found still a little amazing was ' the Director

will then make and issue the final decision." Will he be announcing the recommendations of that panel? I'm a little wary because

we acknowledge that the Director of the Bureau of the Census is a

Presidential appointee.

Can you please address those issues?

Dr. BAILAR. I can certainly try. On the first one, how are we

going to make the adjustment down to the block level, what we

intend to do is to set up a number of strata in which each stratum

would have blocks of like characteristics, and we would make estimates of the undercount from each one of those strata. Then, based

on either using a synthetic adjustment or with some kind of regression analysis, we would be able to use those estimates at that level

to make adjustments down to the block level.

At the block level we would probably do something very similar

to what we do now when we don't know what the tenure of a

household is and we have to decide whether it is occupied or

vacant, and we do a substitution process. We would probably do

something of that nature, though we haven't made that decision

yet.

You have to realize, too, that in most blocks these additions•

unless we really don't have a successful census, the additions at a

64-670 0-86
70

block level would probably be quite small. It could be large at a

city level, but it could be at each block level not very large.

The second question about how we make this decision and that

the Director will make and issue the final decision, at each point

along the way the external group will make recommendations.

Those are open recommendations. They will be widely known,

more than to the Census Bureau. The internal group in itself will

make recommendations, and those things will also be open. Then

the Director will look at those things and make his decision based

on those. That is the role of the Director. He is the Director of the

Bureau.

Mr. GARCIA. Let me thank both of you. Before you go, Dr. Bailar,

just let me say that in the last 3 or 4 days, I have been in the midst

of a big battle trying to make certain that certain forces honor the

confidentiality of what the Bureau of the Census has always stood

for, and I think we have won a battle. So we will keep you posted

on that. But I have asked them to shift gears and not make the

Bureau of the Census responsible for the data as it relates to the

new immigration bill, so that title 13 will continue to be protected

and make sure that the confidentiality is something that is maintained.

I want to thank both of you for being with us today.

Dr. BAILAR. Thank you.

Mr. GARCIA. Our next panel is Dr. Michael Stoto, professor, John

F. Kennedy School of Government, Harvard University; Mr. Carl

Carlucci, executive director of the Legislative Task Force on Demographic Research and Reapportionment; Dr. Eugene Ericksen, professor at Temple University; and Dr. Joseph Waksberg, who is the

vice president of Westat.

We will start panel II off with Dr. Michael Stoto, professor, John

F. Kennedy School of Government.

STATEMENTS OF MICHAEL A. STOTO, ASSOCIATE PROFESSOR OF

PUBLIC POLICY, JOHN F. KENNEDY SCHOOL OF GOVERNMENT,

HARVARD UNIVERSITY; CARL P. CARLUCCI, PH.D., EXECUTIVE

DIRECTOR, NEW YORK STATE LEGISLATIVE TASK FORCE ON

DEMOGRAPHIC RESEARCH AND REAPPORTIONMENT; EUGENE

P. ERICKSEN, PH.D., ASSOCIATE PROFESSOR DEPARTMENT OF

SOCIOLOGY, TEMPLE UNIVERSITY, AND SPECIAL CONSULTANT, NATIONAL ECONOMIC RESEARCH ASSOCIATES, INC.; AND

JOSEPH WAKSBERG, VICE PRESIDENT, WESTAT, INC.

Mr. STOTO. Mr. Chairman, thank you for the opportunity to

appear before you today. The census undercount and adjustment

issue is extremely important for the quality of our national statistics and the policy decisions that depend on them. Your subcommittee is to be congratulated for having the foresight, at a time when

there is little public interest in the 1990 census, to review the

Census Bureau's plans.

In my statement today I will focus on three points: First, appropriate adjustments to the 1990 census require a substantial statistical effort now and in the coming years; second, the Bureau should

develop and employ multiple undercount estimates which rely on

different statistical assumptions; third, decisions about adjusting
71

future censuses should depend on both statistical and political judgment.

After the 1980 census, I was asked to testify in New York about

the Bureau's decision not to adjust that census. Although there was

evidence of substantial undercounts in some areas, there were no

good estimates of how large those errors were. The best available

estimates of the undercount seemed to be no better than the census

figures themselves. Faced with this difficult situation, the Bureau

judged that the adjusted figures would be no more accurate than

the original census figures, and I supported this decision. I believe

that this situation arose because, in the 1970's, the Bureau was

firmly opposed to adjusting the census under any conditions and

did not develop an adequate undercount and adjustment program.

In 1990, we must be better prepared.

Since 1980, the Census Bureau's attitude toward census adjustment has changed in a positive way. The Bureau's current position

is sound: Take the best possible census and prepare to adjust if the

accuracy of the basic census counts and the coverage measurement

program so warrant. Good estimates of census undercount, however, do not come easily or inexpensively. They require a substantial

investment in research and testing and in statistical operations

before, during and after the census. From what I have seen as a

member of the American Statistical Association's Census Advisory

Committee, the Bureau has initiated a strong research program

and it is planning a major effort at the time of the census itself. In

order to ensure a high-quality census in 1990, the Bureau's current

undercount and adjustment efforts must be continued and, perhaps, strengthened.

The centerpiece of the Bureau's plans for 1990 is a coverage

measurement survey, sometimes known as the postenumeration

survey or preenumeration survey or a PEP. Although there were

problems with this method in 1980 and earlier, the Bureau plans

several important changes for 1990. With these changes, the approach represents our best hope for an accurate estimate of the undercount.

However, like all statistical techniques, coverage measurement

surveys relay on assumptions. Thus, they always include some

degree of error.

One way to learn something about the actual magnitude of such

errors is to compare estimates that rely on different assumptions.

If two estimates based on different techniques agree, this in itself is

evidence that the techniques are working well. If the two estimates

do not agree, it is still possible that one of them is correct.

The Bureau has considered taking coverage measurement surveys both before and after the census. This effort would offer some

independence, but both estimates would reflect to some degree the

same sorts of nonsampling errors. Unfortunately, only a limited

number of alternatives to coverage measurement surveys are available for 1990.

Demographic analysis, which has been used since 1950 to calculate undercount estimates, relies on asumptions that are different

from the assumptions of coverage measurement surveys. Demographic analysis, however, only provides estimates for the country

as a whole, not for State and local areas. Thus, the comparison
72

tells us little about the accuracy of the geographical distribution of

the undercount.

The Bureau has considered and experimented with undercount

estimation techniques such as administrative record checks, forward-trace studies, a super census, and systematic observation. As

individual techniques, none of these seems as promising as coverage measurement surveys. However, these techniques do rely on

assumptions that are different from those of coverage measurement surveys. In light of the importance of knowing about the accuracy of undercount estimates, it would be useful to employ as

many independent estimates as possible in 1990.

Between now and 1990, the Bureau plans to build a consensus

about criteria for adjusting the decennial census. This is an important goal that deserves the attention and cooperation of both the

scientific community and government policymakers.

In developing these criteria, the Bureau should remember that

there are many dimensions of accuracy and that a single comprehensive index will be difficult to find. In addition, errors have both

systematic and random components, and these may deserve different treatment. Census adjustment aims to reduce systematic

errors. But because all statistical techniques are subject to some

error, all adjustment methods introduce at least some random

error. Because society is more concerned with systematic error, we

are willing to tolerate some added random error, but how much?

This is not a simple question and the answer goes beyond purely

statistical considerations.

Thus, defining "accuracy" requires a dialog between statisticians

and policymakers. Statisticians have a responsibility to inform policymakers about the characteristics of the available methods, and

policymakers must inform statisticians about their concerns. This

dialog should take place before the results of the census are known,

that is before every decision will have concrete implications for individual States. One way to carry on this dialog is through a series

of simple hypothetical examples designed to illustrate the connection between the measure of accuracy and policy relevant variables. If this were done before 1990, the Census Bureau officials who

must implement any adjustment could learn about the values and

concerns of policymakers in a setting that is not as politically

charged as it will be afterward.

On the statistical side, I prefer general criteria rather than specific. The decision to adjust should be made by the Bureau's statisticians with the advice of others, rather than arbitrary formulas. I

say this because there are no direct connections between easily obtained measures, such as survey response rates, and the many dimensions of accuracy that concern policymakers. No one can say

with certainty that if the coverage measurement survey has a nonresponse rate less than X, a nonmatch rate less than Y, and so on,

the adjusted counts will be more accurate than the census figures

themselves.

Instead, I believe the Bureau should; One, develop a consensus

among policymakers and statisticians about the meaning of accuracy, two, find out as much as possible about the characteristics of

various adjustment estimation techniques and their implications

for the various sorts of accuracy, and three, rely on the judgment
73

of its statisticians, inside and outside the Bureau, to decide whether the adjusted data would provide greater accuracy than the original census figures themselves.

I realize that this approach relies heavily on professional judgment I believe, however, that with a dialog between statisticians

and policymakers in the next few years and before the census, this

approach will lead to the best possible census data in 1990.

Mr. GARCIA. Mr. Carlucci?

STATEMENT OF CARL P. CARLUCCI, PHJX, EXECUTIVE DIRECTOR, NEW YORK STATE LEGISLATIVE TASK FORCE ON DEMOGRAPHIC RESEARCH AND APPORTIONMENT

Mr. CARLUCCI. Thank you, Congressman Garcia. I want to thank

the subcommittee for inviting me here to testify on the subject of

the undercount and population adjustment procedures for the

census.

In 1979,1 was executive director of the Legislative Advisory Task

Force on Demographic Research and Apportionment, and the goal

of that body was specifically and only to monitor the census, to prepare for the release of the numbers and then to reapportion the

State. At that time I was able to participate in the discussion of

undercount and adjustment procedures, and find that the discussion that we are participating in today reflects those same concerns

and the changes between that discussion and this discussion are

not as great as one might expect.

I don't plan to talk about the specifics of the adjustment. I am

not a statistician and I am not a demographer. But I do see that a

number of policy questions are raised, and I would like to try to

separate the question of making an adjustment as a policy and

making a decision about the technology used in the count.

Regarding the first issue, I am not alone in identifying it as an

issue of policy. The NRC group has recognized that this is a policy

decision for a number of reasons: its legal and political ramifications, and because the adjustment procedure would lead to even

more challenges to the census results and possibly delay the release of data. The panel's conclusion that no adjustment methodology would clearly be technically superior dismiss the notion that

the decision to adjust the count could be a purely technical one.

Adjusting the count will almost certainly change the public perception of the decennial census, and that may also change the perception of the Bureau and the Bureau's mission.

What would the impact of an adjusted count be? Well, we have a

substantial body of knowledge about how it would change the distribution of Federal dollars, but the impact of an adjustment on

congressional and State legislative reapportionment is not as clear.

The distribution of funds is an administrative process; it goes on

continually, changes are made repeatedly and in small amounts,

and it could be corrected by administrative action. Apportionment

is a once-in-a-decade decision and involves large units that are indivisible, seats•you can't award fractional seats to States•and it requires legislative and court approval. Funding is by its nature

quantitatively easily subject to adjustment; apportionment is not.
74

According to the work of the Bureau, the impact of an adjustment on reapportionment would be small. But the Bureau points

out that the undercount is not uniform across States and, if the Bureau's methodology is applied using various assumptions, you

produce different results. The Bureau s studies use average underenumeration rates with no variation for communities and make

little or no attempt to deal separately with the white Hispanic population. In fact, in today's testimony the same statement is made.

That those groups have been undercounted, we don't know the

extent of the undercount, and that those groups, especially the

groups that have large numbers of aliens•illegal aliens especially•will be difficult to handle even under the adjustment procedures we have talked about.

The history in regard to the decision to adjust is clear. The 1978

Panel on Decennial Census Plans recommended that an adjusted

count not be used for reapportionment. The 1985 Panel on Decennial Census Methodology recommends that there be only one count

but that adjustment be implemented if there is a responsible confidence that it will reduce differential coverage errors. At a later

point, the panel suggests that time pressures will make it impossible to have an adjusted count available for reapportionment of the

House of Representatives and that a difficult choice must be made.

The panel concludes that the Census Bureau should not make this

choice and that in fact the Congress should.

Regarding the adjustment of the census, the methodologies that

have been discussed today are well-developed, and I want to simply

summarize my comments there by saying that I support the efforts

to expand the post-enumeration activities and the efforts to expand

the study and analysis of the undercount in general. That will be

of benefit to us whether we adjust or not. But the panel believes

that the decisions on methodology should also include a characterization of the purposes to which the census enumeration is used.

The panel does not provide clear criteria for evaluation or selection

of a methodology, leaving the Census Bureau to attempt to make a

determination largely on technical grounds. As a matter of policy,

any determination should be based on the degree to which these

purposes are served.

The result of an adjustment would be that States, if not individual communities, will develop their own calculations and challenge

the methodology. Today, even among the panel, it is clear that

there are different views as to the methodology, its use and its accuracy. Individual jurisdictions will not agree with the panel's position that a jurisdiction's gain or loss of funds or political representation due to error is understood to be always a nonnegative loss

from society's point of view. Furthermore, it is a fact that the undercount, which is caused less because of net national undercount

than because of differential undercoverage by geographic location

and demographic groups, results in distributional inequities that

are the basis for the public support of the adjustment. If these differentials are not remedied by an adjustment, such a procedure

would find little popular support.

I want to make some comments about the practical nature of

dealing with a decision to adjust the count and how it affects the

process.
75

Mr. GARCIA. Please let me interrupt. There is a vote pending,

and I feel you are going to be more than 5 minutes more. So I am

going to recess for about 10 minutes. I will just run over, vote, and

I will come right back.

So we will stand in recess for 10 minutes.

[Recess.]

Mr. GARCIA. There probably will be another vote within the next

20 to 25 minutes, so if you can bear with me, we will just keep running back and forth.

Mr. Carlucci.

Mr. CARLUCCI. Thank you. I wanted to talk about the practical

nature of an adjustment and how such a decision would affect the

process. Because I believe that the result of such a decision made

too early and possibly simply the fact that such a decision is going

to be made could produce a serious negative effect on the census

process. I base that on experience.

In 1980, the Bureau proposed various methods and programs for

the States to participate in a complete count, and at the same time

this discussion was going on. Many areas of the country did not

participate in those programs at all, and for those that did, the results were certainly much less than were hoped for. In New York,

the cochairman of the task force, Assemblyman Miller, was very

interested in the enumeration, supporting the enumeration, and

the problems that New York has with getting a complete count, so

he introduced legislation. And as a matter of fact, he wanted to

come today to attend the hearing; unfortunately, we are preparing

to go back into session to deal with tax changes should they

happen and he was not able to attend. But he was concerned

enough at the time to introduce legislation to support the efforts of

localities and to coordinate their efforts in supporting census enumeration.

The legislation was supported by many of the legislators, the city

of New York, and had support in both Houses, but the bill was

never passed. At the same time the discussion of the undercount

had produced a number of very positive articles in the press. And

in response to a New York Times article, Assemblyman Miller

wrote to them and urged that adjustment be considered apart from

the regular enumeration process and that it not detract from the

effort to achieve an accurate count.

Now, of course, the reaction is that it wouldn't happen. Everyone

says that that will not be the result. In fact, in response to one of

our pleas for help to one of our senior Members of Congress he

identified quite clearly that there would be an adjustment and that

we therefore did not need to participate at the level we were proposing in improving the count on a local level. We had an extensive discussion with him regarding this, and he never changed his

position. He is still a member, and I see no reason for him to react

differently today than he reacted then.

The suggestion that a greatly expanded emphasis on reducing or

eliminating the undercount is something everybody agrees with,

but we see very little effort expended to support it. The prospect of

increased funding, expanded population, snared responsibility, are

extreme measures. It creates a debate but little action.
76

Eventually, when enumeration is just around the corner because

the stakes are so high, the activities of the census will attract the

attention of the public and the press. By that time, the only possible action will be for many politicians and localities to try to push

for a favorable adjustment. I realize that the Bureau is trying to

structure a procedure that will prevent that. It will end up in the

courts. The result will be that the court will ultimately have to

make a decision on those numbers, and, as we have heard today,

both the adjusted numbers and the original counts will be released,

and that will radically change the perception of the census as well

as the perception of the Bureau. It will also change the reapportionment process, and I can't tell you what the result would be.

Thank you.

Mr. GARCIA. Thank you, Mr. Carlucci.

[The statement of Mr. Carlucci follows:]
77

TESTIMOM or

IF. CAT-L T. CASLOCCI

I would like to thank the Subcommittee for inviting

Be to testify cm the subject of the decennial census

undercouct of population and adjustment procedures for the

1990 Decennial Census. As Executive Director of the New

York State Legislative Advisory Task Force on

Reapportionnent 1 was actively involved in New York

State's efforts in support of the 1960 enumeration and

have participated in the discussion of both the impact of

the undercount and of an adjustment. Listening to the

discussion of adjusting the census 1 hear two very

distinct and different questions being asked. The first

is "should we adjust the count" and the second is "can we

adjust the count?" These are both important questions and

deserve to be treated separately.

SHOULD WE ADJUST THE 1990 CENSUS?

Over the years the work of professional demographers

and statisticians has improved the quality of our census

and increased the public's awareness of the undercount

problem. With the publication of AMERICA'S UNCOUNTED

PEOPLE in 1972, COUNTING THE PEOPLE IN I960, and THE

BICENTENNIAL CENSUS the undercount problem has been

analyzed and adjustment consistently recommended, but each

publication recommends different forms and includes a

different set of reservations.
78

Specifically, in COUNTING THE PEOPLE the National

Research Council's Panel on Decennial Census Plans

identified the decision to adjust the census count as a

policy decision and the implementation as a technical

procedure (1). The Panel recognized that the decision to

adjust the census count is a policy decision because of

its political and legal ramifications, and because any

adjustment procedure would lead to even more challenges to

the census results and could delay the release of

acceptable data. The Panel's conclusion that no

adjustment methodology "would clearly be superior

technically" dismissed the notion that the decision to

adjust could ever be a purely technical one 12).

Adjusting the census count will almost certainly change

the public perception of the decennial census, making a

decision to adjust tantamount to a change in our basic

notion of the census and the basic mission of the Census

Bureau.

What would be the impact of an adjusted count? There

is a substantial body of knowledge on how it changes the

distribution of federal dollars to the states, but the

impact of an adjustment on congressional and state

legislative reapportionment is not as clear. The

distribution of funds is an administrative process, it

goes on continuously, changes can be made repeatedly and

in small amounts, and it can be corrected by

administrative action. Apportionment is a
once-in-a-decade decision, involving large indivisible

units, seats, and requires legislative and court approval.

Funding is by its nature quantifiable and easily subject

to statistical analysis and adjustment; apportionment is

not.

According to the work of Jacob Siegel (1975),

analyzing the impact of a population adjustment on the

apportionment of congressional seats, the impact

nationwide would be small. But what about at the

sub-state level? The undercount is not uniform across or

within states and applying Siegel's methodology can

produce different results if varying assumptions as to

undercount rates are used (Carlucci, 1980). While Siegel

estimated that there would be a shift of only two

congressional seats and little or no change in state

legislative reapportionment, he uses average

underenumeration rates with no variation for communities

and makes no attempt to deal separately with the

white-Hispanic population. Siegel states that rates are

not uniform and are higher for minority populations and

probably for urban areas.

While the apportionment of Congress could be carried

out with only national and state populations counts, the

drawing of district boundaries requires counts for smallgeographic areas. A one percent population deviation

standard for congressional districts of 500,000 makes a

small area of 5,000 persons significant (3).
80

Congressional redistricting requires block level data to

meet population equality standards. State legislative

districts have been given more latitude by the court, but

state constitutional rules require levels of accuracy the

make block-level data necessary

Sub-state counts must be precise and agree with state

totals. An adjusted census must produce accurate,

reliable sub-state counts that both the public and the

courts will accept. The acceptance of an adjusted census

count by the court will only happen when the Census Bureau

can produce small area data that use a methodology and

produces a result accepted by the census user communities.

If the result of an adjustment creates two counts, a

population count and an adjusted count, challenges to the

use of either will spring up and the court, out of

necessity, will start determining population. If the

Census Bureau attempted to issue one count for

reapportioning the House of Representatives and an

adjusted count later, which would states be expected to

use in redistricting House seats and their own

legislatures?

The history is clear regarding the responsibility of

the decision to adjust the 1990 census count. In 1978 the

Panel on Decennial Census Plans recommended that the

adjusted count NOT be used for reapportionment (4). In

1985 The Panel on Decennial Census Methodology recommends

that there be only one count, but that "adjustment be
81

implemented if there is reasonable confidence that it will

reduce differential coverage errors (5)." At a later point

the Panel on Decennial Census Methodology suggests that

time pressures will make it impossible to have an adjusted

count available for the reapportionment of the House of

Representatives and a difficult choice will be made. The

Panel concludes that the Census Bureau should not make

this choice and in fact Congress should (6).

CAM WE ADJUST THE CENSUS?

It is clear that there are a number of well developed

methodologies available to adjust the census. Also, there

is no question as to how we should make technical

decisions regarding the development of an adjustment

program and administration of the results of a census

adjustment. All recommendations identify the Census

Bureau and its outstanding professional staff as

responsible for this operation. The Census Bureau is

continuously monitored by an aggressive community of

academic and professional census users who comment on

every technical decision the Census Bureau makes, both as

outside reviewers and as invited advisors to its staff.

The examinations of various adjustment methods

conducted by the National Research Council and the Census

Bureau do raise questions regarding the application of

such methodologies and their results. The complexity of

the methodologies proposed is second only to that of the
actual enumeration and generates differing opinions among

expert statisticians and demographers. I am not expert in

these fields, but 1 will offer an opinion because the

accuracy of various methodologies and the possible scope

of the adjustment are not only questions of accuracy, but

of fairness.

Estimates of the undercount and the methodologies to

adjust are based on data and assumptions that are subject

to error. It is clear that alternative methods for

adjustment can produce significantly different results.

Any adjustment will likely improve the accuracy of the

count for some populations, and resulting allocations, and

thereby decrease the allocations to other populations

(Spencer, 1980).

The Panel (1985) believes that decisions on

methodology should include a consideration of the purposes

which the census enumeration, and any adjustment, are

intended to serve. The Panel does not provide clear

criteria for evaluation or selection of a methodology,

leaving the Census Bureau to attempt to make a

determination largely on technical grounds. As a matter

of policy, any determination must be based on the degree

to which these purposes are served.

Each state, if not individual communities, is likely

to develop its own calculation using sub-state data if

adjustment is attempted, regardless of the methodology or
S3

results. Individual jurisdictions will not agree with the

Panel's position that "A jurisdiction's gain or loss of

funds or political representation due to error is

understood to be always a nonnegative loss from society's

point of view (7)." Furthermore, it is the fact that the

undercount, which is caused 'less because of net national

undercoverage than because of differential undercoverage

by geographic location and demographic group," results in

distributional inequities that are the basis for the

public support for adjustment (8). If these differentials

are not remedied by an adjustment such a procedure would

find little popular support.

The methodologies under consideration all appear to

suffer from the same shortcoming. These procedures work

best for those people with the highest probability of

being counted. Those who are most often missed will still

be missed. Central cities with large minority populations

typically have the highest gross omission rates. These

populations are most likely to be missed in a

mailout-mailback census. In addition low income

individuals are less likely to generate the paper trail

that would lead to their location or the accurate

estimation of their numbers.

That there is an awareness of this lack of uniformity

and the need for adjustments to be tailored to local

characteristics is not enough. The test of the

acceptability of any methodology is the extent to which it
84

adjusts for members of "hard-to-count populations."

HOW WILL AN ADJUSTMENT AFFECT THE 1990 ENUMERATION?

Of a more practical nature is the question of how a

decision to adjust the count will affect the census

process. I suggest that the result of such a decision,

either made too early or even offered as a solution to the

undercount problem, would produce a serious negative

effect.

In 1980 the Census Bureau proposed various methods

and programs by which the states could support the work of

the Census Bureau in achieving as complete a count as

possible. At the same time the discussion of an

adjustment was raging.

Many areas of the country did not even participate in

the Census Bureau's Local Review program and the results

of the program overall were far less than hoped for. In

New York, Assemblyman Melvin Miller, co-chairman of the

body charged by the state legislature with monitoring

census activities in New York in preparation for the 1990

reapportionment, introduced legislation to support and

coordinate the enumeration support and outreach programs

of localities (9). While the legislation was supported by

many legislators and the City of New York, the bill never

passed.
85

In 1980. in response to a February article touting

adjustment, Assemblyman Miller wrote to the New York Times

and urged that adjustaent be considered apart from the

regular enumeration procedure, so that it would not

detract from an effort to achieve the most accurate count

possible. Of course the reaction is that no such thing is

intended and would never be the case. In fact, in

response to our petition for help to one of the senior

members of the New York Congressional delegation, we were

told that an adjustment was imminent and was much

preferable to investing time and money on improving the

basic count.

The solution is a greatly expanded emphasis on

reducing, if not eliminating the undercount. This

suggestion evokes nods of agreements and some laughs, but

little serious action. The prospect of increased funding,

expanded participation, shared responsibility or extreme

measures leads to endless debate, but little or no action.

Eventually, when the enumeration is just around the

corner, because the stakes are so high, the decennial

census machine will attract the attention of the public

and the press. By that time the only possible action many

politician and localities will be able to take is to push

for a favorable adjustment to the census count for their

area. The pressure on those making the adjustment

calculations will be enormous. The result will surely be

a court challenge of incredible proportions. Given the
86

shortage of funds and the existing conflicts that threaten

the 1990 census effort, I question whether or not the

Census Bureau, the future of the decennial census, and the

nation's confidence in our census could withstand such a

challenge.

Notes.

1. NRC, 1978, p. 111.

2. NRC, 1978, p.105.

3. Wollock cites a "de minimis" rule which has been

interpreted as 1% or less by 80% of the States

(Grofman, 1985, p. 84).

4. NRC, 1978, p.112.

5. NRC, 1978, p.278.

6. NRC, 1985, recommendation 7.6, p. 297.

7. NRC, 1985, p. 279.

8. NRC, 1985, p. 280.

9. New York State Assembly, 1980 Legislative Session,

Bill No. A.8858-A
87

Mr. GARCIA. Dr. Ericksen?

STATEMENT OF EUGENE P. ERICKSEN. ASSOCIATE PROFESSOR,

DEPARTMENT OF SOCIOLOGY, TEMPLE UNIVERSITY, AND SPECIAL CONSULTANT, NATIONAL ECONOMIC RESEARCH ASSOCIATES. INC.

Dr. ERICKSEN. I am very happy to be given the opportunity to

testify before you. I have submitted my testimony along with an article, "Estimating the Population in a Census Year,' coauthored

with Joseph B. Kadane. which expands upon the views in my testimony.1

I would like to summarize my points very briefly because

some of them have been made by other speakers.

One is that an important feature of the error in the census is

that it is systematic. By design and as implemented, like any good

survey, the census is designed to count those whose homes are

clearly recognizable as residential structures at identifiable addresses, who are willing to cooperate with the Government by providing the requisite information, who are sufficiently well educated

and proficient in English, and who are members of a well-defined

family unit. When these conditions are not met, the likelihood of

census error is greater. This situation has existed for every census

and going back as far as 1940 it has been evaluated. Every time we

find the same result. Blacks are under-counted at a higher rate

than whites. In 1980, because of advances in evaluation methodology, we also learned that Hispanics are undercounted at rates similar to blacks and that blacks and Hispanics are missed at higher

rates when they live in central cities like New York City and Chicago than when they live elsewhere.

Now, in 1970, according to demographic analysis, the differential

between the black undercount and the nonblack undercount was

6.1 percent. In 1980, by the same analysis the differential was 5.9

percent. I don't believe that it is possible to develop a census-taking

methodology which will eliminate that differential given the rules

of a free society that we have here in the United States. As I see it,

we have two alternatives to get the count right. One is the coverage improvement alternative. According to the General Accounting

Office, $342 million were spent to improve coverage in the 1980

census. The National Academy of Sciences evaluated special programs which cost $100 million and they found the following results:

Twenty-eight million dollars were devoted toward improving address lists. These procedures got favorable results as 16.4 million

people were added to the count as a result of those procedures.

That comes to $1.71 per capita. However, for coverage improvement procedures that were implemented during the data collection

process itself, according to an NAS panel $69.7 million were directed toward that, 2.6 million people were added, that is over $26 per

added person. That is not a cost effective way to budget for a

census.

Available methods for adjustment exist and the Bureau has used

them in the past. To give you one example, I have done a great

deal of research on methods of calculating postcensal estimates for

1 Retained in hearing file
88

revenue sharing and other purposes. These are ways in which the

census count is adjusted, hopefully to take into account changes in

population and income since the census count. The technical issues

involved in calculating those estimates are no easier, probably

harder, than the technical issues involved with adjustment. Somehow the political situation is different so that the technical scrutiny which is applied to the adjustment situation is not applied to

the calculation of these estimates.

I believe that the Director of the Census Bureau for the 1980

Census, Vincent Barabba, summed up the situation quite succinctly

when he wrote, as follows:

If, in fact, the Bureau does all of the things it plans to do, and the enumeration is

still differential, then the dilemma arises. The decision about what you do in that

case is a political decision.

Now I believe that many of the important political decisions will

be made now. This is because, as the Census Bureau has stated

many times, there is a long period of time that is needed for planning purposes. The important funding decisions will be made soon,

and the Census Bureau should be given all the money that it needs

to develop an adjustment program. Money is better spent on developing a methodology for good adjustment now than toward the illusory goal of coverage improvement in 1990. The Congress should

make it clear that the adjustment methodology has its mandate

and, indeed, that the Congress expects an adjustment to occur in

order to get the final count correct.

Now I would like to make a couple of comments to respond to

things that were said, if I may, by a couple of the previous speakers. I think that a couple of the points that Mr. Carlucci made, 1

would not exactly say the same thing that he did. It is my understanding of the PEP methodology that it is designed in a way that

it can calculate separate estimates for Hispanics and for different

types of communities.

Second, in my view, the adjustments that are made are conservative. It is true that the current population survey and probably the

post-enumeration survey in 1990 will miss the same kinds of people

that the census misses. However, the adjustment that is made on

the basis of those people who are found will go sufficiently far in

the right direction to make a useful difference.

Finally, I would like to say that in terms of the litigation it is not

easy for a locality to bring a lawsuit against the Census Bureau. In

my view, if the Census Bureau has a professionally acceptable adjustment strategy, that will stand up in almost any court in the

United States. The courts give the Census Bureau a great deal of

discretion.

Thank you.

Mr. GARCIA. Thank you, Dr. Ericksen.

[The statement of Mr. Ericksen follows:]
89

TESTIMONY OF EUGENE P. ERICKSEN

TO BE GIVEN BEFORE THE HOUSE COMMITTEE

ON POST OFFICE AND CIVIL SERVICE.

SUBCOMMITTEE ON CENSUS AND POPULATION,

JULY 24. 1986

My name is Eugene P. Encksen. I am a survey

statistician and social scientist with over 20 years experience of working with census data. Currently, I am an

Associate Professor in the Department of Sociology at Temple

University and a Special Consultant to National Economic

Research Associates, Inc. I am also a co-author, with

Professor Joseph B. Kadane, of the paper, "Estimating the

Population in a Census Year," published in 1985 in the

Journal of the American Statistical Association. This

paper, which I submit along with this testimony, describes

my views concerning methods of data collection and estimation on the Decennial Census. I respectfully also refer the

subcommittee members to other testimony which I have submitted to the subcommittee on this subject on other occasions.

I am here to make three points: (1) that regardless of how much money and effort are expended on the

traditional headcount, a differential undercount is inevitable in the American Census; (2) that coverage improvement

procedures, aimed at simply redoubling traditional headcounting efforts, do not solve the problem and indeed can
90

increase the amount of error in census data; and (3) that

methods of estimation exist by which an imperfect census can

be adjusted to bring the count closer to the truth. I

conclude that for the 1990 Census, we should direct funds

toward the development and implementation of appropriate

adjustment techniques rather than toward so-called "coverage

improvement".

1. Error in the Census Is Inevitable

The Census is a survey that provides and can only

provide an estimate of the true population. It is inevitably fraught with error in the collection of data that

necessarily results from census procedures. Inequity occurs

when, as is invariably the case, the errors are greater in

some places than others. In the United States, errors due

to undercounting are greatest in central city neighborhoods

inhabited by minority populations. These errors have caused

these cities and their states to be shortchanged in terms of

political representation and fund allocations.

Problems giving rise to error beset any attempt to

collect survey data. Such problems are present in magnified

form in an undertaking as massive as the census. This is

especially true because the majority of census workers at

many levels are temporary employees without prior experience

in collection of data. At peak periods during the 1980
91

Census, the Bure*-.: emclryed afccct 27S.OOO tt^orary eaployees in its field offices sad about: 6.300 t<•JIHITJ eaployees

in its three specialized in in •ill isiliis It is

extremely difficult to find people qualified tc be enumerators in poor urban areas, and the time and funds available

to train thea for the 30b axe I 111 111 Quality control

problems inevitably cause errors in census data as the

Bureau itself has recognized in evaluation studies that

follow the Decennial Census (e.g.. ER-60 Series after the

1960 Census. PHC(E) Series after the 1970 Census, and PHC-BO

Series after the 1980 Census).

An important feature of the errors in the Census

is that they are systematic. By design and as implemented.

the Census is best suited to count those whose homes are

clearly recognizable as residential structures at identifiable addresses, who are willing to cooperate with the

government by providing the requisite information, who are

proficient in English, who are educated and who are members

of a well-defined family unit. Where those conditions are

not met, the likelihood of census error is greater. Problems can be identified at each step of the census process.

The first step is the preparation of address

lists. In urban areas the Census Bureau relies on commercial address lists which it then updates through field

checks. A study by the General Accounting Office has shown
92

that the commercial lists are much worse in poor central

city neighborhoods than they are elsewhere. Updating the

lists through field checking is also difficult in these

neighborhoods, which are often dangerous and in which multiple housing units at the same address are frequently disguised and hard to spot without contacting the residents.

Additional errors occur when census forms are

mailed to those addresses included in the listings. In

order for a household to receive a form, it must not be lost

in the mail for any reason. The mailing label must be

affixed properly, the address must be legible, and the form

must be handled properly by the Post Office. In apartment

buildings or other multi-unit structures where mail is

received collectively, the residents must distribute the

questionnaires correctly among themselves for each household

to receive its form. In areas like the South Bronx, where

there were thousands of broken mailboxes and where the questionnaires often did not include the name of the addressee

as part of the mailing address, many people did not receive

census forms in 1980.

Assuming a questionnaire is received, someone in

the household must open it, answer it correctly, and return

it to the Bureau. The questionnaire may go unopened or it

may be opened but then lost. The questions may not be

understood by the respondent and thus may be incorrectly
93

answered, particularly if the respondent is poorly educated

or if the respondent s native ianguage is not English. Even

if the questionnaire is correctly answered and Bailed back,

it may be lost in the Bail. Assuming the fora is received

by the Census Bureau, it Bust be correctly recorded and

tabulated. Inevitably in this process, forms are lost or

destroyed and the data on soae are Bisrecorded or Bistabulated.

If the household does not return the fora, an

enumerator is sent to visit the address and attempt to

collect the form. Depending on the crime rate and ease of

finding the address in the area, this is a job of varying

difficulty. Where the mailback rate is higher, this job is

easier. While a low mailback rate does not mean that people

inevitably will be missed, it certainly makes it more likely

because the job is so much harder. The task was substantially more difficult in the Bedford-Stuyvesant area of

Brooklyn, where fewer than half the population mailed back

their census forms, than in a middle-class suburban area

where the mailback rate was 90 to 95 per cent.

In some cases, the enumerators are not able to

find any household residents. In such instances, the

enumerator is told to seek out a neighbor, building janitor,

or similar outside source to obtain information about the

unit. Such people have imperfect information, and less than
94

ideal motivation to supply it. Moreover, many enumerators,

perhaps in frustration, simply "invent" their own information in a well-known procedure called "curbstoning".

Finally, the quality of the follow-up, like the quality of

the address lists, is adversely affected by the use of a

large number of temporary employees as enumerators.

The foregoing points are not made in criticism of

the Census Bureau, but merely to show that it is impossible

accurately to count every person and that it is the most

disadvantaged members of our society who, by the nature of

the Census, are the ones predominantly missed. Indeed, the

Census Bureau recognizes the existence of most of these

problems, and has attempted to remedy them. The remedies,

however, are themselves imperfect solutions. Since these

imperfections are most likely to result in errors in situations where it is most difficult to count people, the errors

are cumulative. It is a general principle of survey

research that checks need be made at every step of the

process of data collection. This means checking the completeness of the address list, making sure that all forms

are returned, checking the validity of data on every form,

and making sure that all data are correctly processed

through the computer. As the data base grows larger, the

proportion of resources needed to be devoted to quality

control grows larger. When the survey attains the size of a
95

Decennial Census, the task of complete quality control

becomes monumental.

2. The Geographical Distribution of the Undercount

The Census Bureau evaluated the undercount by

means of its Post Enumeration Program (PEP) in 1980. Best

estimates indicate that 10 to 13 million persons were

omitted from the Census, 6 million others were erroneously

enumerated and 3 million additional person-records were

created by computer by a method known as imputation. This

resulted in a net national undercount of 1 to 4 million

persons, concentrated in central cities. The PEP produced

separate estimates for 16 large central cities like New York

and Chicago. In these cities, the combined omission rate

was estimated to be 9.9 per cent, compared to 5.0 per cent

elsewhere. The rate of erroneous enumeration was 2.6 per

cent, comparable to the 2.9 per cent observed elsewhere.

The net result was an undercount of 6 per cent in the cities

and 1 per cent in the rest of the country. As shown in the

table below, the differential undercount was especially

great for Blacks and Hispanics, who were missed at rates

exceeding ten per cent in the cities.

The 1980 Census was the first one for which this

type of geographic detail was presented on the undercount.

However, the racial pattern is similar to that observed on
96

censuses since 1940. For example, in 1970 the Bureau used

demographic methods to estimate an undercount of 7.6 per

cent for Blacks and 1.5 per cent for Whites, a differential

of 6.1 percent.

Undercount Rates for Three Ethnic Groups

in Central Cities and Elsewhere, 1980

Group

Location Blacks Non-Black Hiepanics

Central Cities 11.3% 10.3%

Elsewhere 5.5% 4.5%

Total 7.2% 5.9%

The racial differential for 1980 as measured by demographic

analysis was 5.9 percent, a scant improvement. It is this

differential which is so intractible, resisting the best

efforts of the Census Bureau to obtain complete and equitable counts. On the basis of the results for 1980 and

earlier years, we can expect the same pattern of differential undercounting to occur in 1990 and later.

3. Coverage Improvement Campaigns Do Not Solve the Problem

The 1980 Census was by far the most expensive in

American history. According to a report of the General

Other Total

1.5% 6.0%

0.2% 1.0%

0.3% 1-6%
97

Accounting Office, the inflation-adjusted cost of counting

one person in 1980. $4.72, was more than twice the 1970

cost, $2,30. According to a panel of the National Academy

of Sciences, approximately $100 million was invested in a

variety of coverage improvement programs, a substantial

increase over the $13 million (1980 dollars) spent in 1970.

The panel concluded that coverage improvement programs

directed toward improving address lists improved the count

in a cost effective manner, as 16.4 million people were

added at a cost of $28.1 million, $1.71 per capita. However, coverage improvement programs attempting to add

population during the data collection period were less

effective, as only 2.6 million people were added at a cost

of $69.7 million, $26.81 per capita. There were at least

two reasons for this negative result.

One is that Census Bureau personnel, already

overwhelmed by the routine tasks of data collection and

tabulation, simply had no time to carry out the coverage

improvement procedures correctly. This is illustrated by

the Nonhousehold Sources Program in which lists of licensed

drivers and welfare recipients were supposed to be compared

to census registers, with omitted persons added to the

count. An evaluation by the General Accounting Office found

that this program was usually not fully implemented. The
98

NAS panel found that only 130,000 persons were added at a

total cost of $9.8 million $75.54 per capita.

A second problem arose form the Bureau's policy

regarding sampling. In 1970, the Census Bureau carried out

a National Vacancy Check in a sample of 13,500 vacant

housing units. It found that 8.5 per cent were in fact

occupied, so the Bureau randomly changed a corresponding

proportion of all vacant housing to the status of "occupied." Just over one million persons were added to the

count at a per capita cost of 21 cents. In 1980, the Bureau

decided that sampling procedures should not be used, and

carried out the National Vacancy Check on all housing units

initially listed as vacant. Here, 1.7 million persons were

added, but at a per capita cost of $21.12, one hundred times

the corresponding cost in 1970.

There is ample evidence that, in addition to its

failure to reduce the omission rate substantially, the

coverage improvement programs caused erroneous enumerations

to increase. For example, without a corresponding check of

households initially classified as "occupied" the National

Vacancy Check was bound to inflate the count. The NAS Panel

found that several coverage improvement programs could cause

erroneous enumerations, and this is consistent with the

results of two Census Bureau studies.
99

The first, a study of housing unit duplications,

found that approximately two million people live in housing

units that were counted twice, often in rural areas where

enumeration district boundaries are less clear and street

addresses lacking. The second, a more comprehensive estimate of erroneous enumerations, found that they were much

more likely to occur on follow-up activities than on the

initial mailout-mailback. Among persons counted during the

follow-up period when enumerators collected the data, one in

fourteen turned out to be erroneous enumerations.

There are many potential explanations for this

high rate of error. One is that the period of data collection lasts so long, stretching into the fall of 1980 in many

areas. Given the high mobility of the American population,

the same family could be counted twice in different places

(either because it moved or had two residences) with the

family being unaware of it because information was provided

by different family members or by neighbors. Another

explanation is simple fabrication, occurring when enumerators are under great pressure to get forms completed to meet

Bureau deadlines.
100

4. Adjusting the Counts

The Bureau's studies show a consistent pattern of

undercounting, with Blacks and Hispanics missed at higher

rates than Whites, and with minorities living in central

cities missed at higher rates than minorities living elsewhere. The extensive and systematic nature of the undercount consistently shown by the Bureau's own studies makes

clear that adjusting the count will improve the accuracy of

the results. Indeed, the Census Bureau has itself used

adjustment in other contexts. For example, the methods used

by the Census Bureau to calculate post-censal estimates of

population and per capital income involve the same matching,

regression, and demographic estimation methods Professor

Kadane and I have advocated for adjusting the Census.

Moreover, when in 1970 the Census Bureau needed to adjust

Census estimates of per capita income for very small places,

it used the same regression method described in our paper.

Much of the reason the Census Bureau hesitates to

adjust the Census for the differential undercount is

political. As Vincent Barabba, Director of the Census

Bureau during the 1980 Census, said:

"If, in fact, the Bureau does all of the things it

plans to do, and the enumeration is still differential,

then the dilemma arises.
101

"The decision about what you do in that case is a

political decision." ("A Right to be Counted",

(American Demographics 44 at 46 (1979))

The Census has been taken without adjustment for the disproportionate undercount for 200 years, and with all the

scrutiny focused on the count, the Bureau hesitates to make

a change unless it feels it has the clear blessing of

Congress.

The Census Bureau has not directed sufficient

research effort toward developing good adjustment methods,

and it is likely that current and future research can be

very fruitful. As Barbara Bailar, the Associate Director

for Statistical Standards of the Census Bureau stated in the

New York trial, "We have been doing matching studies off and

on over a period of 30 years ... I think if we had been

working on matching and really devoting all kinds of

research necessary to learn how to do matching well, we

would probably be a lot further ahead." In my opinion,

money allocated by the Congress to further these research

efforts would be money well spent.

5. Conclusion

I draw two conclusions from the foregoing:

(1) that it is foolhardy to invest large sums of money

toward the illusory goal of coverage improvement; and

(2) that the Congress should give a clear mandate to the

64-670 0-86-5
102

Census Bureau to make its best estimate of the population,

which will necessarily require an adjustment. Indeed,

Congress should make clear that the Bureau's failure to

correct for differential undercounting would constitute a

failure to perform its responsibilities.

We have learned from the 1980 Census that regardless of the amount of money invested in coverage improvement, millions of people will be omitted, and that these

people are likely to be Black, Hispanic, and live in cities.

The problem is made worse by the fact that coverage improvement procedures cause additional errors to occur, further

reducing the quality of the data.

Demographic analysis makes it plain that Blacks

are missed at much higher rates than nonBlacks. Survey

results make it clear that Hispanics are missed at rates

similar to Blacks, and that persons living in central cities

are much more likely to be missed than persons living

elsewhere. Any reasonable adjustment shifting population

shares into these central cities will produce a more accurate and equitable estimate of America"s population

distribution, and hence will improve the situation. It is

time for the Congress to assure that the next Census will be

significantly more cost-effective and accurate than those of

the past.
103

Mr. GARCIA. Mr. Joseph Waksberg.

STATEMENT OF JOSEPH WAKSBERG, VICE PRESIDENT, WESTAT,

INC

Mr. WAKSBERG. Thank you, Mr. Chairman.

For at least the last four decades, the Census Bureau has carried

out programs that evaluated the quality of the decennial censuses

and provided most of the available information on the undercount.

Other speakers at today's hearings are discussing the relationship

of evaluation to adjustment. I am concerned that the attention to

adjustment procedures may erroneously create the impression that

the sole or primary purpose of evaluating census data is to assist

an adjustment and that if adjustment is not planned, then there is

no need for an evaluation program. It would be unfortunate if a

decision is reached on the conduct of the 1990 census based on this

line of reasoning.

There are good reasons why censuses were evaluated long before

the recent drive to adjust the census results and they still apply. I

would like to describe the purposes of census evaluation and research programs, the uses to which they have been put and future

needs for comparable research.

Evaluation and research programs have three major objectives.

The first is to provide information to the Bureau staff on the kinds

of errors arising in censuses and the cost effectiveness of potential

methodological improvements. Second, to provide to users of census

data estimates of errors for their knowledge in understanding the

strengths and the limitations of census data. And third, to conduct

experiments and analyses that evaluate specific procedural steps

and devices. It should be noted that there are many experiments

that can only be carried out during the course of a census, either

because they require very large sample sizes which are only available in the census or they involve public perception which is affected by the publicity and other forms of public education during a

census.

I would like to illustrate the importance of the practical applications of the research with examples of specific major uses of past

research. The list of examples I will give is far from exhaustive. I

have selected the more important uses of research, with particular

attention to the ones with which I am personally familiar. Other

examples and more detailed information on the ones I will mention

are included in my written testimony.

First, let me go on to the impact of evaluation and research on

the methods used in taking the census itself. Since 1950 there have

been fundamental changes in the methods of census taking, based

largely on analyses of prior research. Virtually all of the many specific procedures introduced in 1960 or later are direct outgrowths of

lessons learned in evaluation studies. Let me give some examples.

One of the key elements in current censuses is the use of selfenumeration. Reliance on self-enumeration was first introduced in

1960, and it was a direct outgrowth of what was learned from a

1950 research study referred to as the Enumerator Variance Study,

as well as two other experiments. The attempt to increase the level
104

of self-enumeration was a major motivation in developing plans for

a mail census.

In connection with coverage improvement, earlier evaluations indicated that in spite of all attempts to improve the completeness of

the count in conventional types of censuses, it was still at an unsatisfactory level. This led to research on alternative methods of improving coverage, including establishing an independent mailing

list for mail census, using the post office to update and correct it,

and other similar methods.

Similar analyses were involved in the choices of coverage improvement programs used in the 1980 census. A variety of methods

were introduced in 1970 and were intensively studied in the 1970

evaluation program. The effectiveness of each method was an important ingredient in the development of the 1980 coverage improvement program, especially in the determination of the geographic scope of these programs.

Another illustration is the continuation of the collection of data

on a sample basis. The 1950 census was the first one to rely significantly on sampling for the collection of basic data formerly collected in the complete census, and also for the preparation of detailed

tabulations. However, there were a number of restrictions that

were imposed on the determination of what items could be collected on a sample basis. The decision to expand sampling in 1960 was

essentially based on analyses of the 1950 data. The extensive use of

sampling was retained in 1970 and 1980. Sampling permitted sizable reductions in costs and in the time needed for the preparation

and publication of census reports.

Evaluation programs indicated the need for changes in the wording of questions for such major items as labor force, educational attainment, and other subjects. The revised formats for these items

have been adopted as the standard in most private statistical organizations as well as the Bureau's current surveys.

Evaluation data are also used as a standard to test whether anticipated advantages from changes in procedures actually materialize. For example, the 1960 results on coverage were an essential

factor in assessing whether the test studies of the mail-out/mail-in

census procedure for 1970 produced hoped for improvements in

quality.

Let me go on to uses of the research and evaluation studies by

users of census data. The results of the evaluations of past censuses

have been widely distributed by the Census Bureau and are commonly used by serious analysts in interpreting statistical data. The

research program has also had an important effect on statistical

methods used by other statistical organizations, both government

and private, in the United States and abroad. I will give some examples.

First of all, let me point out the importance of the evaluation

data in keeping congressional oversight committees such as yours

apprised of what is going on in the censuses. Evaluation data are

essential in keeping Congress informed about the quality of each

census and whether funds allocated for improvements in census

methods are having their desired effects. I find it difficult to visualize how the subcommittee could carry out oversight functions effectively in the absence of data on the quality of the census from the
105

evaluation and research programs. I should also note that because

of the widespread use of census data for Federal fund allocation,

other congressional committees are also concerned with and have

raised questions about the quality of census data.

Another important group of users are persons concerned with

vital statistics data. Birth rates and death rates in the United

States are calculated by using birth and death registration data as

the numerators in the computations and census counts as the denominators. Errors in the denominators can distort the results as

well as those in the numerators. The National Center for Health

Statistics, which prepares and publishes vital statistics data, indicates to its users the importance of using census evaluation data in

interpreting the results.

Census Bureau analysts are themselves important users of evaluation data. The Census Bureau regularly updates the census population counts for the United States in its intercensal population estimates program. They produce updated figures every year•in

fact, every month.

For a number of technical reasons, the census counts are first adjusted for underenumeration during the preparation of the estimates and later reduced by equivalent levels. The procedure is referred to as inflation and deflation.

Data on undercounts also enter into the projections of the population in future years made both by the Census Bureau and other

agencies. Some of the projections have critical policy implications.

For example, Social Security Administration's estimates of the

aged during the next 50 years is a major input in determining

Social Security programs, and even small errors may have serious

implications.

The monthly statistics on employment, unemployment, and related labor force items are anchored to the Census Bureau's intercensal population estimates by age, sex, and race, as are annual data

on education, migration, fertility, income, and almost all other

Census Bureau social statistics. Undercount estimates are thus involved in virtually all of the Bureau's population estimates.

Other organizations involved in the collection of statistical data,

including universities, market research firms, those in public opinion measurement, and government statistical agencies in other

countries, have all benefited from the research that was an outgrowth of the census evaluations. The results of the research have

affected statistical methods in a wide variety of areas, including

question wording, geographical classification, data processing, comEuter systems, as well as the methods of data collection carried out

y these organizations. As an example, it can be noted that the

Bureau was the first statistical agency to experiment with and

make use of computers for data processing and with an automatic

data entry system. This revealed the potential and versatility of instruments that have become standard tools and are now taken for

granted.

Let me make a few final comments. I have attempted to describe

major uses to which the Census Bureau's research has been put in

the last two or three decades. There is, of course, no assurance that

the research conducted in 1990 or future years will have the same

impact on statistical methods as the studies that have been carried
106

out in the past. However, one should not take the attitude that

nothing further needs to be known about the conduct of censuses.

The Census Bureau is staffed with competent statisticians and it is

reasonable to assume that they will continue the Bureau's long tradition of being in the forefront of statistical methods.

I must say I was somewhat hesitant about appearing before this

committee to defend research. It seemed almost self-evident that in

our stage of society, a large organization cannot afford not to conduct research. The industrial companies that have gone out of business because they have not kept abreast of the latest technology

provide ample evidence of the need for research. However, I realize

that when there is pressure of impending budget cuts there is

always a temptation to sacrifice the future for the present, and I

thought it would be useful to remind the committee of the shortsightedness of such a policy.

Mr. GARCIA. Thank you very much.

[The statement of Mr. Waksberg follows. Also included are his

responses to written questions:]
107

IN THE U.S. CENSUSES OF POPULATION AND HOUSING

Presented before the U.S. House of Representatives

Subcommittee on Census and Population

July 24, 1986

Joseph Waksberg, Vice President, Westat, Inc.

1. General Purposes of Census Evaluation and Research

Today's hearing of the U.S. House of Representatives'

Subcommittee on Census and Population is concerned with the

undercount in past population censuses, and the potential for

partially compensating for it in 1990 by adjusting the census

results. In order to adjust the census figures, it is necessary

to have detailed data on persons who are missed in the census.

For at least the last four decades, the Census Bureau has carried

out programs that evaluated the quality of the decennial censuses

and provided most of the available information on the undercount.

Other speakers today are discussing the relationship of evaluation to adjustment. I am concerned that the attention to

adjustment procedures may erroneously create the impression that

the sole or primary purpose of evaluating census data is to

assist in adjustment and that if adjustment is not planned, then

there is no need for an evaluation program.

It would be extremely unfortunate if a decision is

reached on the conduct of the 1990 census based on this line of

reasoning. There were good reasons why censuses were evaluated

long before the recent drive to adjust census results, and they

still apply. -I would like to describe the purposes of census

evaluation or, more broadly, census evaluation and research
108

programs, the uses to which they have been put, and future needs

for comparable research.

The evaluation and research programs can be considered

as having three major objectives:

1. To provide information on the kinds of errors

arising in censuses and the cost effectiveness of

procedures being used to the Census Bureau's staff

for their use in planning future censuses. The

focus here is on methodological improvements and

an improved design of the total system.

2. To provide to the users of census data estimates

of errors for key census statistics and related

information on the components of the errors. This

is primarily for their knowledge in understanding

the strengths and limitations of census data.

3. To conduct experiments and analyses that evaluate

specific procedural steps and devices. The

purpose is the same as for the first objective, to

examine ways of improving the conduct of future

censuses. It should be noted that there are many

experiments that can only be carried out during

the course of a census, either because they

require very large sample sizes which are only

available during a census or they involve public

perception which is affected by the publicity and

other forms of public education during a census.

I would like to illustrate the importance of the

practical applications of the research with examples of specific

major uses of past research and evaluation programs. Before

going on, I want to make some comments on my choice of examples

of research effects. First, the list of examples discussed is

far from exhaustive. I have selected the more important uses of

the research, with particular attention to the ones with which I

am personally familiar. Secondly, I have concentrated on results

of research studies of population and housing censuses. Similar
109

examples could be provided for the Census Bureau's economic

censuses, but they do not appear to be relevant to these

hearings. Finally, I have paid more attention to uses made by

the Census Bureau than to other consumers of the data. This is

not because I believe they are more important, but because they

illustrate the importance of the research more clearly.

2. Impact of Evaluation and Research on Methodology of

U.S. Censuses

Since 1950, there have been fundamental changes in the

methodology of census-taking, based largely on analyses of the

results of prior research. Virtually all of the many specific

procedures introduced in 1960 or later are direct outgrowths of

lessons learned in earlier evaluation studies.

The most important of these new procedures are

described in the balance of this statement. However, it should

be emphasized that a list of such examples cannot adequately

cover the various ways that knowledge of the strength, weaknesses

and limitations of data are used in interpreting the results and

in making decisions about plans for future censuses. The facts

regarding the quality of statistics that emerged have become

important background information for the Census staff and it is

as difficult to delineate the occasions in which the knowledge is

used as to itemize the uses of census statistics themselves.

Several other aspects of the evaluation programs should

be noted. The first concerns the relation between recent and

earlier studies. Much of the 1980 program duplicated types of

information collected in previous decades. There were
no

essentially three reasons for the duplication: (a) At each

census, the Bureau developed hypotheses on how to improve the

measures of quality previously used. Consequently, the Bureau

has been able to progressively sharpen the measuring tools and

obtain more precise measures of quality for these items; (b) It

was felt desirable to obtain readings on measures of census

quality at each Census rather than to assume a constant level of

quality; (c) The changes in census methods introduced at each

census are expected to produce changes in quality, generally

improvements. It is important to measure quality under the

changed conditions, and to try to ascertain the extent to which

improvements occurred.

This final reason has an important bearing on the

reasons for the evaluation programs. In a sense, one of the

purposes of each evaluation program is to ascertain whether the

census is moving in the right direction in adopting the

procedures utilized.

The second aspect is that direct implications of the

results of a particular program cannot always be anticipated in

advance. Some experimental studies are geared to test the

feasibility of a specific change in method. However, many of the

other studies are initiated purely to get background information

on quality of the statistics. It is only when analysts are

concerned or- even shocked at some of the results that new methods

are explored to overcome the weaknesses. Evaluation programs

will probably always have, at least partially, the characteristic

of investigating statistical quality, initially to determine if

any problems exist, and then only if conditions warrant them to

propose procedural changes.
Ill

Finally, the need for detailed evaluation does not

imply that nothing would be known about the quality of census

statistics if research using such methods as reenumerative

surveys, matching, analysis of the census operations, etc., were

not conducted. A considerable amount of insight into problem

areas is obtained by comparison of census totals with results of

other statistical systems, such as vital statistics, social

security records, and earlier censuses. However, with minor

exceptions, evaluation programs are the only existing devices for

obtaining reasonably precise measures of the extent of gross

error, the causes of error, and whether or not any progress is

being made in many aspects of census methods. This knowledge

permits attention to be focused on the important problems and

leads to the development of improved methods.

The most important procedural devices used in recent

Censuses that have developed from the evaluation programs follow:

Self-enumeration and Mail Censuses

One of the key elements in current censuses is the use

of self-enumeration. Reliance on self-enumeration was first

introduced in 1960. It was a direct outgrowth of what was

learned from a 1950 study referred to as the Enumerator Variance

Study and two other experimental projects carried out in 1950.

The 1960 Response Variance Study was conducted to get a second

reading on this contribution to total variance, and to ascertain

whether the anticipated reduction in variance due to the use of

self-enumeration really occurred. The findings confirmed the

results expected, and the 1970 and 1980 Censuses were even more

dependent on self-enumeration than 1960. The attempt to increase
112

the level of self-enumeration was a major motivation in

developing plans for a mail census although, as indicated below,

there were other reasons also.

Coverage Improvements through Use of Mailing Lists.

Post Offices, and Other Special Programs

Earlier evaluations indicated that in spite of all

attempts to improve coverage in conventional types of censuses,

it was still at an unsatisfactory level. This led to an

examination of alternative methods of improving coverage. The

1960 Census contained an experiment to determine whether Post

Office resources could be used to improve coverage, and a similar

smaller scale study was conducted in 1957. Subsequently, the

procedure of creating an independent mailing list and using the

Post Office to update and correct it was developed as the basis

of the mail census planned for 1970 and repeated in 1980. Once

the plans for the mail census were developed, it became clear

that there would be many additional advantages • for example,

extension of self-enumeration, greater flexibility in geographic

coding, smaller enumerator recruitment needs, and the ability to

concentrate more attention in difficult enumeration areas.

However, the initial impetus was related to the facts emerging

from the coverage analysis.

Similar analyses were involved in the choices of

coverage improvement procedures used in the 1980 Census. A

variety of methods were introduced in 1970 and were intensively

studied in the 1970 evaluation program. The effectiveness of

each method was an important ingredient in the development of the
113

1980 coverage improvement program, especially in the determination of the geographic scope of these programs.

Continuation of the Collection of Data on Sample Basis

Although the 1940 census collected a limited amount of

information on a sample basis, the 1950 Census was the first one

to rely significantly on sampling for the collection of basic

data formerly collected in the complete census, and for the

preparation of detailed tabulations. However, there were a

number of restrictions that were imposed on the determination of

what items could be collected on a sample basis.

In the 1960 and subsequent Censuses, the restrictions

on the use of sampling were almost completely removed, and all

items except those virtually required to define and count the

population, and the housing items needed for block statistics,

were collected on a sample basis. Three factors brought about

this extension of the use of sampling:

a. The evaluation programs • in particular the

matching of Census with current sample survey

results, and the various response variance studies

• provided evidence that for most census

statistics, the introduction of a moderate

sampling error would have only a minor effect on

_- the total error. This mostly came about because

the reduction of response error due to the

elimination of the interviewer component of

response variance was sufficient to compensate for

the introduction of sampling error.

b. The sampling procedures were carried through in

the field operations reasonably well. This is not

to say that some biases in sample selection did

not turn up. The biases, however, were not great
114

enough to have a serious effect on most uses of

the data.

c. There were sizeable reductions in cost and in the

time needed for preparation and publication of the

census report.

The decision to expand sampling in 1960 was essentially

based on an analysis of 1950 data. The results of the I960

evaluation program were examined to determine if the revisions in

census methods introduced in 1960 had enough effect on the

relationship of sampling error to total error to require any

changes in our attitude on the use of sampling. No changes in

approach to the use of sampling seemed called for, and the

extensive use of sampling was retained in the 1970 and 1980

Census. As in 1950, analyses of the composition of the sample

indicated some biases in the sample selection, but these were not

great enough to compensate for the great gains in efficiency

brought about by the use of sampling.

Selection of Households for the Census Sample

Studies made of the composition of the 25 percent

sample in 1960, including an analysis of differences between

sample and nonsample households, indicated the presence of some

small but troublesome biases. The 1960 sample apparently had too

few one-person households and too many with three-persons. Also

there seemed to be shortages of secondary individuals and certain

other types of household members.

One of the motivations for considering a mail census

was that it permitted a largely computer-selected sample, thus
115

avoiding the possibility of the enumerator biasing the sample by

modifying the order of listing.

Measurement of Housing Quality

In developing plans for a mail census, one of the

problems that was discussed involved information on quality of

housing. In 1960 and 1950 this item was based on the

enumerator's visual observations of the structural soundness of

the housing units. A three-way classification was used in 1960

• sound, deteriorating, or dilapidated.

No promising method of obtaining comparable information

through self-enumeration turned up in several pilot studies. At

first, it appeared possible that if an alternative was not

developed, a mail census might be rejected by the need for

housing quality data. An analysis of the response variance for

this item indicated that the enumerator's observations were

completely unstable, arising from the subjective nature of the

enumerator's judgments on quality. Considering the crude nature

of the interviewer's measurement of this item, it became clear

that obtaining accurate data on quality should not be considered

a major goal of the census.

Effect of Evaluation Program on Question Wording

The evaluation programs indicated the need for changes

in the wording of questions in later censuses. For example, the

first test of self-enumeration and a mail census was carried out

in an experimental study in 1950. The evaluation of labor-force
116

items in this experiment suggested that the battery of questions

used to establish labor force status in the 1950 Census was not

an effective device for self-enumeration. Respondents apparently

would not follow correctly a skip-pattern whose logic and

significance were not clear to them. On the other hand, they did

skip questions which appeared irrelevant to them, even when the

fine print on the questionnaire told them not to. Considerable

revisions in the set of questions were made for 1960 on the basis

of this analysis that led to successful collection of labor force

data by self enumeration. The revisions were retained in the

1970 and 1980 Censuses.

Another example is in the collection of data on

educational attainment. Evaluation of earlier censuses indicated

many respondents were not clear on the distinction between

highest grade attended and highest grade completed. The

resulting errors in reporting were largely eliminated when the

wording was altered; first a question was asked on highest grade

attended, followed by a separate inquiry on whether that grade

was completed. This format has been followed on all recent

census and has been adopted as the standard on the Bureau's

current surveys.

Use of Evaluation Results in Testing New Procedures

As discussed in previous sections, evaluation results

provide measures of the strengths and the weaknesses of census

data, which become the motivation for developing improved methods

for conducting censuses. When these alternate methods are

pretested in the field, the evaluation data become a standard of

whether the anticipated advantages actually materialize. For
117

example, the 1960 evaluation results on coverage were an

essential factor in assessing whether the test studies of the

mail-out-mail-in census procedures produced the hoped-for

improvements in quality. Coverage evaluation programs were

carried out as part of the tests conducted between 1960 and 1970.

Only by comparing the results with the 1960 coverage evaluation

figures, could effectiveness of the new procedures be compared

with population and housing unit counts obtained by the older

census methods.

3. Needs for Evaluation and Research Results bv Users of

Census Data

The results of the evaluations of past censuses have

been widely distributed by the Census Bureau. They are commonly

used by serious analysts in interpreting statistical data. The

research program has also had an important effect on statistical

methods used by other statistical organizations, both Government

and private, in the U.S. and abroad. Some examples follow.

Congressional Oversight Committees

Evaluation data are essential in keeping Congress

informed about the quality of each census, and whether funds

allocated for improvements in census methods are having their

desired effects. The staff of the Subcommittee on Census and

Population is, of course, more familiar with data they use to

make judgments about the Bureau's programs than I am, but I find

it difficult to visualize how the Subcommittee could carry out

oversight functions effectively without information from the
118

evaluation and research program. I should also note that because

of the widespread use of census data for Federal fund allocation,

other Congressional committees are also concerned with the

quality of census data. One of my last official duties before I

retired from the Census Bureau in 1973 was to testify before the

House Committee on Labor and Education. The Committee was

holding hearings on a renewal of the education act which provides

for allocation of funds to school districts on the basis of the

number of below-poverty school children. The Committee wanted to

make sure that the quality of the census figures was sufficiently

high for the data to serve as a basis for allocation. Without

the evaluation results, there would have been no way of replying

to the questions that were raised.

Vital Statistics

Birth rates and death rates in the U.S. are calculated

by using birth and death registration data as the numerators in

the computations and census counts in the denominator. Errors in

the denominators can distort the results. In a recent report by

the National Center for Health Statistics on Mortality (Vital

Statistics of the U.S., 1978, Vol. II - Mortality), there is a

detailed discussion of the census undercount and its effects on

rates. The appendix notes that the impact of the undercount "...

can be of several types: (1) Effects on levels of observed

rates, (2) effects on differences among groups; and (3) effects

on the levels and group differences shown by summary measures

such as age adjusted rates and life expectancy." The appendix

describes how users can adjust the vital rates on the basis of

evaluation data, and provides some examples of the effects. A

particularly striking example given is the fact that for deaths
119

from homicide, "... the age-adjusted death rate for black males

would decrease by 10.4 percent from 72.8 to 65.2 deaths per

100,000 population."

Preparation of Intercensal Population Estimates and

Population Projections

Census Bureau analysts are themselves important users

of evaluation data. The Census Bureau regularly updates the

census population counts for the U.S. in its intercensal

population estimates program. For a number of technical reasons,

the census counts are first adjusted for under-enumeration during

the preparation of the estimates and later reduced by equivalent

levels. The procedure is referred to as inflation-deflation.

Intercensal population estimates using evaluation

results have also been developed by other analysts. Historic

data on the U.S. population by age, sex, and race based on census

data that were adjusted for undercoverage, were recently prepared

and described in an article in the November 1984 issue of

Demography•

Data on undercounts also enter into the projections of

the population in future years. Some of the projections have

critical policy implications. For example, Social Security

Administration's estimates of the aged during the next 50 years

is a major input in determining social security programs, and

even small errors may have serious implications.

Intercensal population estimates for cities, counties,

and other small areas are prepared by both Governmental agencies
120

and commercial organizations. These data are used for a variety

of purposes, such as fund allocation and determination of market

share and marketing effort. Census data are virtually always key

ingredients in the estimates, and the research on the undercount,

on the number of undocumented aliens, and on other issues

affecting the quality of the census are factors used in

understanding the limitations and improving the quality of the

data.

Labor Force Data

The monthly statistics on employment, unemployment, and

related labor force items are anchored to the Census Bureau's

intercensal population estimates, by age, sex, and race. The use

of Census evaluation results in the intercensal figures are thus

reflected in unemployment, employment and related labor force

data. In addition, the fact that under-enumeration is not

identical from census to census causes discontinuities in the

labor force time series. The evaluation data are helpful in

distinguishing between real changes over time, and those

artificially created by the changes in census coverage.

Intercensal Population Estimates for Revenue Sharing

Purposes

An illustration of an unexpected application of census

research involves Revenue Sharing. Federal funds are allocated

to about 39,000 local governments on the basis of a formula that

includes the population of the localities. The Census Bureau

regularly updates the population figures so that the allocations

reflect the current situations.
121

During the 1960's, the Bureau started experimental work

on the possibility of using Internal Revenue Service micro

records in the evaluation of income and other selected census

data, when the need for population updates for the many small

governmental areas in Revenue Sharing arose, it quickly became

apparent to Census analysts that the IRS data were the only

existing source of information on migration into and out of these

areas, and the IRS files became a key component of the estimation

procedure. The prior research permitted a program to be

developed quickly. It considerably advanced the production of

the needed data, possibly by up to two years.

As a by-product of both the original research and the

Revenue Sharing estimates, the methodology also improved

statistics on migration that are used for many other purposes.

Census Research Uses bv Other Statistical Organizations

Other organizations involved in the collection of

statistical data • including universities, market research

firms, those in public opinion measurement, and government

statistical agencies in other countries • have all benefited

from the research that was an outgrowth of the Census

evaluations. The results of the research have affected

statistical methods in a wide variety of areas, including

question wording, geographical classification, data processing,

as well as the methods of data collection carried out by these

organization. As an example, it can be noted that the Bureau was

the first statistical agency to experiment with and make use of

computers for data processing and with an automatic data entry

system. This revealed the potential and versatility of
122

instruments that have become standard tools and are now taken for

granted.

4. Final Remarks

I have attempted to describe major uses to which the

Census Bureau's research has been put in the last two or three

decades. There is, of course, no assurance that research

conducted in 1990 or future years will have the same impact on

statistical methodology as the studies that have been carried out

in the past. However, one should not take the attitude that

nothing further needs to be known about the conduct of censuses.

The Census Bureau is staffed with competent statisticians and it

is reasonable to assume they will continue the Bureau's long

tradition of being in the forefront of statistical methods.

I was somewhat hesitant about appearing before this

Committee to defend research. It seemed almost self-evident that

in our stage of society, a large organization cannot afford not

to conduct research. The industrial companies that have gone out

of business because they have not kept abreast of the latest

technology provide ample evidence of the need for research.

However, I realize that when there is pressure of impending

budget cuts there is always a temptation to sacrifice the future

for the present, and I felt the responsibility of reminding the

Committee of the shortsightedness of such a policy.
123

WESTAT

An Employee Owned Oesear-cn C3"Do^aton

-.- o.o •£*»..

August 4, 1986

The Honorable Robert Garcia

Chairman

U.S. House of Representatives

Committee on Post Office and Civil Service

Subcommittee on Census and Population

219 Cannon House Office Building

Washington, D.C. 20515

Dear Mr. Garcia:

This is in reply to the questions you raised in your July 25

letter regarding plans for evaluating the 1990 census.

Following the tradition of good statistical practice, I

would like to start off with some caveats. It's about 13 years

since I left the Census Bureau. During this period, attendance

at professional meetings and my work on the Panel on Decennial

Census Methodology of the Committee on National Statistics have

kept me informed about the Bureau's activities in a general way,

but it's likely that I am not aware of many of the details in the

research currently carried out. As a result, some of my

suggestions may be superseded by recent developments. Comments

by the Bureau's professional staff on these subjects should

therefore be taken into account in considering my recommended

actions.

Keeping these qualifications in mind, I will attempt to

answer your questions.

1. Possible improvements in methods of evaluating completeness

of the count

a) Although your question referred to improvements the

Bureau's staff could make, my first suggestions refer

to action outside the Bureau's scope. Demographic

analysis is one of the methods used to estimate

national undercoverage. Improvement in statistics on

immigration and emigration would greatly enhance the

accuracy of demographic analysis. The recent report on

immigration and emigration data by the Committee on

National Statistics indicates that there are serious

deficiencies in the quality of these data, not only for

illegal entrants, but also for other types of migrants.

I assume that the Bureau cannot do much to improve
124

these data, but possibly the Subcommittee can use its

influence in this regard.

Another deficiency in demographic analysis is its

inability to estimate coverage of the Hispanic

population. Part of the reason is that the quality

problems of migration data affect the Hispanic

estimates more severely than most other population

groups. However, even if these problems were solved,

demographic analysis would still be unable to estimate

the Hispanic undercount because of the lack of birth

and death statistics for all Hispanics. Data on the

number of births and deaths come from birth and death

registrations. The National Center for Health

Statistics coordinates efforts for standardization of

vital statistics records among the states. I

understand that there has been considerable success in

the attempt to persuade state agencies responsible for

vital statistics record to include an item on Hispanic

origin on the birth and death certificates. Although

only 24 or 25 states include this item, they are the

ones with the largest concentrations of the U.S.

Hispanic population, and they account for 85 to 90

percent of births and deaths. A revised standard for

the certificates is scheduled to be effective in 1989,

and it is hoped that the number of states requesting

this information will increase.

For purposes of coverage evaluation, it is necessary to

have almost complete coverage in the vital statistics

records. Perhaps the Subcommittee can assist by urging

states to adopt this standard.

I should note this last point would be of future help

to demographic analysis rather than of current use. It

probably would require 30 or 40 years of birth and

death data before the kind of demographic analysis

applied to the black and the total white population

could be used for Hispanics.

b) In regard to sample survey methods of coverage

evaluation, the main technique relied on by the Bureau

in 1980, my suggestions refer to emphasis applied to

certain components rather than to different ways of

operating.

A very high response rate in the evaluation studies is

essential to their success. It is important that the

Bureau's field organization which is responsible for

the actual data collection understands this and is
125

responsive to the research staff's insistence on the

follow-up necessary to achieve high rates.

Accurate matching of survey and census data is also

necessary. I understand the Bureau has had

considerable success in developing automated matching.

However, the ability to determine whether unmatched

census cases should actually have been included in the

post enumeration sample survey is partly dependent on

the clarity and uniqueness of the areas used to select

the sample. The 1980 Census used the CPS sample.

Although CPS is very efficient for the purposes it was

designed to satisfy, the fact that its sample does not

use easily identifiable geographic boundaries keeps it

from being an ideal instrument for census coverage

evaluation. The Bureau has been exploring the use of

city blocks and equivalent areas for sample selection.

I think continuing research on this approach should be

pursued vigorously. The Bureau should also examine

related techniques, such as using complete buildings as

sample segments.

c) Methods of combining the best features of demographic

analysis with post enumeration surveys should be

examined. Some of this is currently being done at the

Bureau, but I suspect that more emphasis would be

useful. For example, the problems of the accuracy of

demographic analysis (at the national level) are mostly

due to the quality of immigration and emigration data.

These do not have much effect on the black population.

I suggest exploring the possibility of using

demographic analysis for the black population and

survey methods for others.

2. Budget for Evaluation

I hope the wording of the second question does not imply

that the Bureau's research program for the 1990 census would be

restricted to evaluating the population count. As I tried to

make clear in my prepared statement for the Subcommittee's

hearing, there are important past uses and future needs for

research on other aspects of the census.

I assume the "ballpark figure" you asked for refers to a

budget figure. I cannot give any specific dollar amount since I

have not kept up with current salary levels, overhead rates, data

processing costs, etc. My reply, therefore, relates the 1990

budget for research to the 1980 amounts. Also, the caveats I

mentioned at the beginning of the letter are particularly

relevant to this part of my response.
126

I think it would be reasonable to start off with the

assumption of a 1990 budget for research at the same level as

1980 updated for inflation, and see whether the current condition

call for an increase or decrease in that amount. An early step

should be to estimate costs separately for studies of the

completeness of the count and for other research projects since

only the former are involved in adjustments. For noncoverage

research, the 1980 level is probably appropriate, unless the

Bureau indicates pressing needs for specific areas of research

that did not exist for 1980.

The scope of the coverage studies should depend on whether

or not the data will be used for adjustment. If the data are to

be used for adjustment, the sample size needs to be large enough

to provide considerable geographic detail, possibly state data.

More limited geographic detail, and thus a smaller size, is

sufficient if adjustment is not planned. Approximate sample

sizes for the coverage evaluation can then be compared to the

1980 program in preparing the budget estimates. One point should

be noted. The 1980 coverage estimates used CPS rather than an

independently selected sample. As I indicated in response

question 1, I think the Bureau should seriously consider using

blocks or other geographic areas as the sample base rather than

relying on CPS. I assume this will tend to increase the costs

over 1980.

If a decision on whether to adjust is not reached until 1990

then there is uncertainty on whether detailed geographic data for

adjustment will be necessary. The plan outlined at the

Subcommittee hearing by Dr. Bailar indicates the final decision

will be delayed until an examination of evaluation data is made

in 1990. The sample size, scope of the program, and budget have

to be fixed long before then. Some compromise will have to be

made on the sample size and budget.

3. Congressional Support

Probably the most useful kind of support Congress can give

to the Census Bureau is for your Subcommittee to continue the

kind of hearings it has been holding, which provide a forum for

interested parties to express their views on census plans. The

Census Bureau has, of course, advisory committees as well as more

informal ways of ascertaining the views of users of census data

and other concerned social scientists. However, the importance

of Congressional hearings obviously make participants consider

their positions much more carefully than attendance at other

kinds of meetings.

Other areas of support I can think of that would be helpful

to the Bureau are only partly within the scope of the
127

Subcommittee's jurisdiction, but the Subcommittee may influence

decisions by making its views known. One area is the budget. I

think it's important that there are funds available for research

and development.

Secondly, the requirement for early release of the

population counts may be a major stumbling blocks in the ability

to adjust the data. Delaying the release dates by one or two

months might substantially improve the feasibility of adjustment.

I realize that the early release is tied to redistricting plans

and any delay may have serious implications. Perhaps the

Subcommittee could look into the possibility of some relief from

the tight deadline.

A third issue of a somewhat different character relates to

the proliferation of suits to revise the census figures after the

1980 census. Preparing testimony occupied as inordinate amount

of time for some of the most talented and productive staff

members and seriously detracted from attention to important

planning activities. Anything that could be done to reduce such

drains on the staff would be quite helpful.

I hope these responses are useful to your Subcommittee's

work. If I can help in any other way, please let me know.

Sincerely,

Joseph Waksberg

Vice President

JW/eg
128

Mr. GARCIA. Mr. Carlucci, you have just heard the Census

Bureau testify that they do the best job that can be done in actually counting people, but it is still not enough. In fact, the National

Academy panel says that anything more they do in this field might

make things worse. Yet, you seem to be saying that they should do

better and that deciding to adjust might make it hard for them to

actually get people's help.

What do you think the Census Bureau should do about the undercount of poor people and minorities?

Mr. CARLUCCI. Well, I can't say I have a clear answer, and I don't

think that anybody believes that the methodology that the Bureau

is developing•and I am glad to hear that other people think that

that methodology will be so effective. I don't think anybody is suggesting that there is a better way to adjust. I am suggesting that

there are still problems. An adjustment may improve the count of

those groups that have had the severest undercount, but the methodology still will produce a differential adjustment, that certain

groups will receive less of an adjustment than others.

Part of the reason is that those groups and the areas they live in

are more difficult to count, and to some extent it is the responsibility of those areas to participate and to support the Bureau in doing

that. I realize how difficult it is to get local support, and I guess

what I am saying is that the promise of an adjustment will extinguish whatever effort exists for local support.

I don't know, and I am not saying that is enough of a reason to

not adjust. I am not saying that, as a matter of fact. I am saying

that is something you have to take into account in the timing. NAS

has said that the decision should be made early, and it is concerned

about the Bureau's intent to make the decision at a later date.

That timing is certainly going to have an impact on how people

participate and how that adjustment is seen.

Mr. GARCIA. Thank you, Mr. Carlucci.

This question is adressed to both Professor Stoto and Professor

Ericksen. In your statements, you both mention and you urge that

the Congress itself play more of a role and be more involved in the

question of the adjustment. Now I will say to both of you, having

spent 20 years of my life in an elected office and having lived

through five reapportionments, that I think you are wrong in the

sense that I just don't think that we are the proper judges of adjustments because it seems to me that we would slant everything

in our favor. I mean I just think that would be the trend. Nobody

apportions himself out of a district. I would appreciate it if you

would be kind enough to comment.

Mr. STOTO. That certainly is a very difficult question. I think

that it is important to stress the fact that the input, the policy and

political input should come before 1990, before the figures are in. I

think it is only human nature that once the figures are in people,

Congressmen would find it very difficult to think objectively about

criteria.

On the other hand, perhaps I can give an example of something I

believe would be helpful before 1990. As I mentioned in my statement, there were various different aspects of accuracy and there is

no simple agreement on them. For instance, would it be an improvement, if suppose that we knew with certainty that the census
129

was perfectly accurate in 49 States, but had an undercount of 10

percent in one State, and we could change that situation and make

it to one in which there was a 50-50 chance that every State would

be either 2 percent up or 2 percent down? That kind of an adjustment is one that substitutes random error for systematic error.

It may be helpful for the statisticians to propose hypothetical situations like that to Congressmen and say, would you be willing to

adjust•do you regard that as an improvement? And as long as the

State of Massachusetts is not that one, perhaps people could make

judgments about what constitutes improvement.

Mr. GARCIA. You are aware of how congressional seats are apportioned, are you not?

Mr. STOTO. Yes, I am.

Mr. GARCIA. In terms of orders of priority based on population,

and, Mr. Carlucci, maybe you can correct me if I am wrong, I think

New York received a 34th Member of Congress on a 432 or 433

order of priority. I think we may have won but I think Indiana was

one of the States that lost a seat. It seems to me based on what we

are talking about now an adjustment in that one State, as it relates

to the priority as we select after the first go-round may lead to further questioning and lawsuits. In 1980, long after the census was

completed and the States were reapportioned, I met with a delegation from the State of Indiana. I met with the Governor, the U.S.

Senators, and the full congressional delegation•because they lost a

seat. Yet they thought that they should have kept the seat, and I

think they argued for an adjustment. And rightfully so. I mean I

can't blame them.

But the point is that, as chairman of the subcommittee during

the last census and now as chairman prior to the forthcoming

census, I would like to state to both you gentlemen that I think the

most that Congress can do is exactly what we are doing now. That

is, monitoring the census and making sure that we have these

hearings on an ongoing basis to let them know this committee is

holding up to its responsibility as it relates to making sure we

know what they are doing.

But I would not advocate that we play a role outside of this committee, or maybe the Appropriations Committee which handles

Commerce and the Bureau of the Census. So I say that to you because it just seems to me that I would not like politicians to get '

involved in the process. I just think we should stay out of it, except

to do that we are doing.

Mr. STOTO. I agree with you. If I can make this short statement, I

believe that it is important for this subcommittee to take up these

matters, and I think this is the appropriate forum.

Mr. GARCIA. Yes, right.

Dr. Ericksen, if you can, for a minute, I guess, respond to that.

And, Mr. Waksberg, if there is anything you would like to add for

another minute, and then I am going to have to run to vote. And I

don't think I am going to come back, so we will close the hearing. I

have some questions, though, that I would like to submit to you,

and I would appreciate very much if you can get your responses

back to staff so that we can complete the record on this hearing.

Dr. Ericksen.
130

Dr. ERICKSEN. I want to state very clearly that in my comments I

had no intention that the Congress should have anything to do

with the decision about adjustment after the data are in. You can

be sure that if Florida had the 436th seat and Indiana had the

435th the lawsuit would have gone the other way around.

Mr. GARCIA. That is right.

Dr. ERICKSEN. What I meant was, the Congress has a choice, and

that choice will be made sometime in the near future, and this is

where Congress plays a very important role. In allocating the

budget for the 1990 census, if the Congress goes the route of saying

we want you to have coverage improvement programs and we are

going to allocate another $342 million or more, then, in not allocating money for the development of an adjustment methodology, that

is how the decision not to adjust will essentially be made, in my

opinion.

On the other hand, if you say to the Census Bureau, we want you

to emphasize the development of an adjustment methodology, we

know from the 1980 census, a lot of the coverage improvement programs didn't work, then you will be furthering the cause of an adjusted and a more accurate census.

Mr. GARCIA. Thank you. Mr. Waksberg, is there anything you

would like to add for another 30 seconds?

Mr. WAKSBERG. Well, I want to make just one comment about

the concern about different levels of undercount and how to change

them. Looking at adjustment by itself as if it were the only issue

that gets involved is probably looking at a very small part of the

issue. The Bureau, in its planning, makes decisions which affects it

for example, in the $300-odd million spent for coverage improvement programs. If you look at where that goes, it is obviously to

improve coverage where coverage needs improvements•in the

inner cities of the big cities, some of the types of rural areas, and

so on. These decisions made by the Bureau themselves are intended

to reduce the level of differential undercount. And I am not sure I

see why thinking of adjustment changes the perception of what the

Bureau should be doing there.

Mr. GARCIA. I would like to thank all of you very much. I would

hope that you would stay in touch with this subcommittee because

obviously there is a great deal that has to be done, and this subcommittee is really concerned about 1990. We are working on it, as

you can see. We are starting early, and we are not going to stop.

We have been at the process now for about a year and we will continue.

[Whereupon, at 12 noon, the subcommittee was adjourned, to reconvene subject to the call of the Chair.]

8587 95

64-670 (136)