# Presentation: 347406

## An Evaluation of the Accuracy of U.S. Bureau of the Census County Population Estimates

**An Evaluation of the Accuracy of U.S. Bureau of the Census County Population Estimates**

*by*
*Dean H. Judson,*
* **University of Nevada, Reno *
*Carole Popoff,*
* **Decision Analytics, Inc.*
*Michael J. Batutis,*
* **U.S. Bureau of the** **Census, Population Division*

## This study:

*This study:*

*As primary data collection becomes more expensive, administrative records will become the primary collection point - work needs to be done to ensure data quality and  to develop corrections for errors and biases.*
*We examine the sources of data used in the Administrative Records method for intercensal county estimates, and further develop our model of error in the estimates process.*
*We use the results of the model to develop a correction factor usable within the current county estimation system and to propose specific improvements to the current system.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Sources of Administrative Records Data used in the County Method

*Sources of Administrative Records Data used in the County Method*

*Births and deaths:*
*supplied by state vital statistics, or *
*supplied by NCHS in lieu of state reporting.*
*Group Quarters counts:*
*supplied by the state or the entity that collects the counts, or*
*last year’s figures if the current count is not provided by the state.*
*Foreign immigration:*
*INS intended place of residence (for documented immigrants), and*
*fixed percentage applied to an estimate of undocumented immigrants (for undocumented immigrants).*
*65 and older:*
*Medicare records.*
*Net migration:*
*IRS tax returns matched year over year.*
*See paper for substantial additional detail.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Our Basic Approach

*Our Basic Approach*

*We argue that:*
*U.S. administrative records data can be used as an indicator of the population of interest, but not ****(currently)**** as a direct count.  (Finland appears to have mostly solved problems with direct use.)*
*Administrative records data collection methods suffer from biases caused by the data collection mechanism itself:*
*Conceptual differences between data and the object being measured;*
*Reporting lags;*
*Tendency to geographically misallocate events; and*
*Inaccurate or incomplete record matching.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Our Basic Approach (cont.)

*We hypothesize reasons why the data collection mechanism may not represent the underlying population.*
*Using these hypotheses, we identify county-level variables which would be indicative of:*
*The underlying condition (i.e. rapid change, high Medicare ****non****-enrollment); *
*The group unlikely to be covered by the administrative record (i.e., those in poverty, counties without hospitals) or*
*other “reporting problems” (i.e. lag in GQ reporting)*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Example of How a Bias Is Created in the Migration Rate

*Example of How a Bias Is Created in the Migration Rate*

*The net migration rate can be positive or negative.*
*If an unemployed non-filer gets a job in another county, moves and files next year, then the tax return is not matched.*
*This family is not added to the number of movers, thus*
*the net migration rate in the county of origin is too high and the county of destination is too low.*
*See tables 1 and 2 in the paper.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Hypothesized Sources of Estimation Error  for Group Quarters Counts

*Hypothesized Sources of Estimation Error  for Group Quarters Counts*

*A person files a tax return and is matched, but subsequently enters a group quarter and is also counted there (or the opposite).  *
*There will be an over- (under-) estimate for county of origin.*
*Limited group quarters updating by the FSCPE may not capture change in this population.*
*Using last year’s numbers will create over-estimates under conditions of decline, or under-estimates in conditions of growth.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Hypothesized Sources of Estimation Error For Natural Increase 

*Hypothesized Sources of Estimation Error For Natural Increase *

*If a birth mother must go to another county because there is no hospital in her county of residence, then,*
*the birth may be recorded in the county where the hospital is located.  *
*Therefore, *
*there will be an over-estimate in the county where the hospital exists, and*
*an under-estimate in the county of birth mother’s residence.*
*Deaths can experience the same phenomenon; however**,*
*since the number of births > number of deaths we expect an incorrectly high level of natural increase  and an over-estimate.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Hypothesized Sources of Estimation Error for Foreign Immigration

*Hypothesized Sources of Estimation Error for Foreign Immigration*

* **A county’s proportion of foreign persons will change but illegal immigrants are allocated using percentages from the 1990 census.*
* *
*This results in an under- or over-estimate depending on the direction of change in the proportion of foreign born in a particular county.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Hypothesized Sources of Estimation Error for Population 65 Years of Age and Older

*Hypothesized Sources of Estimation Error for Population 65 Years of Age and Older*

*The propensity to sign up for Medicare will be conditioned on knowledge of the program.  For example,** *
*The foreign born, less educated, or others with language or cultural barriers are less likely to enroll.*
*Under-enrollment is not uniform across counties; we predict under-estimates in counties with high proportions of persons 65 and older and not enrolled.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Methodology - Data

*Methodology - Data*

*1990 Census counts are compared to the Administrative Records Method using 1990 estimates generated by Davis (1994).*
*We calculate Algebraic Percent Errors (ALPEs) for each county.*
_*The key is that this reflects magnitude AND direction of error*_*.*
*Bureau of the Census USA Counties CD-ROM data are extracted.*
*We identify socio-economic variables as indicators of the source of error.*
*The two sources of data are matched by FIPS codes**.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## A Note on Census Undercounts

*A Note on Census Undercounts*

*The key error in the Census count is undercount**.*
*We obtained from Post Enumeration Survey materials the adjusted population counts.*
*We created a corrected ALPE (CALPE) measure using adjusted population counts.  Correlation between ALPE and CALPE = .968*
*We re-ran analyses using CALPE as the error measure, and compared results to ALPE.*
*At this time, we feel confident that our hypotheses for bias are almost entirely independent of the undercount correction.**  *

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## A Description of the Models

*A Description of the Models*

*The final model was developed by adding groups of variables indicative of the hypothesized bias for** **each type of administrative record.*
*Model 0 contains 4 regional variables, total 1990 population, and the change in population from 1980 to 1990, and represents a “null” model.*
*Model 1 contains the null model and adds:*
*birth/death misallocation indicators*
*group quarters misreporting indicators*
*Over 65 underreporting indicators*
*Immigration indicators*
*Net internal migration bias indicators*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Results are consistent with our hypotheses

*Results are consistent with our hypotheses*

*Model 1 significantly improves upon Model 0.*
*In the presence of the additional indicator variables, the estimated coefficients on variables in the null model tend toward zero.*
*When we use 1990 population corrected for undercount, little change in coefficients occurs, with one exception:*
*“**Indians in poverty” is strongly positively correlated with ALPE, holding other effects constant; its effect drops to near zero in the CALPE model.*
*Other  effects remain substantially the same.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Summary of Substantive Results

*Summary of Substantive Results*

*Consistent with other studies, counties with rapid change tend to have estimation errors opposite the direction of change.*
*Counties with a high rate of natural increase tend to be overestimated. *
*Rural counties with no hospital tend to be underestimated.*
*Counties with a large proportion of the population in group quarters tend to be underestimated, and this effect is strongly exacerbated if the county is also growing.*
*A notable exception to #4 is counties with a high percentage of its population in prison.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Summary of Substantive Results (Cont)

*Counties with a high proportion of SSI enrollees tend to be overestimated, but if the county has a high proportion of the 65 and over population that is not enrolled, the county tends to be underestimated.*
*Counties with a high percentage foreign born tend to be underestimated.*
*Counties with a high percentage of the population in poverty tend to be overestimated.*
*Counties with a high percentage of native American Indian population tend to be underestimated.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Summary of Substantive Results (Cont)

*Our model implies a correction factor:*

- 1+XB is the correction factor.

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com

## Implications for Post-2000

*Implications for **Post-2000*

*General conclusion: Using an administrative records data series for any estimation method requires careful consideration of “program effects” creating systematic biases.*
*Specific proposals for post-2000:*
*Our results provide a correction factor for the estimates series.  After further careful study, this correction factor should be incorporated into the estimates process. *
*In particular, we should use the 1990 correction factor to generate a 2000 estimate and determine whether the corrected estimate is better or worse than an uncorrected estimate.*
*Vital statistics geocoding needs to be improved at all levels.*
*FSCPE Group Quarters update and reporting is very important.*
*A Medicare coverage study would be useful for assessing predictors of undercoverage.*

- Contact:  Dean H. Judson, Ph.D. e-mail demogecon@aol.com