Using American Community Survey Data to Improve

Estimates from Smaller U.S. Surveys through

Bivariate Small Area Estimation Models

William R. Bell, U.S. Census Bureau

Carolina Franco, NORC at the University of Chicago

November 3, 2021

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 1 / 15
Introduction

We investigate the potential for borrowing strength from ACS

estimates via bivariate small area estimation models to reduce

variances of estimates from smaller U.S. surveys.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 2 / 15
Introduction

We investigate the potential for borrowing strength from ACS

estimates via bivariate small area estimation models to reduce

variances of estimates from smaller U.S. surveys.

The population quantities measured by ACS and the smaller survey

must be related, but are not assumed to be the same.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 2 / 15
Introduction

We investigate the potential for borrowing strength from ACS

estimates via bivariate small area estimation models to reduce

variances of estimates from smaller U.S. surveys.

The population quantities measured by ACS and the smaller survey

must be related, but are not assumed to be the same.

The bivariate modeling is very simple!

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 2 / 15
Introduction

We investigate the potential for borrowing strength from ACS

estimates via bivariate small area estimation models to reduce

variances of estimates from smaller U.S. surveys.

The population quantities measured by ACS and the smaller survey

must be related, but are not assumed to be the same.

The bivariate modeling is very simple!

No covariates from auxiliary information are used!

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 2 / 15
Introduction

We investigate the potential for borrowing strength from ACS

estimates via bivariate small area estimation models to reduce

variances of estimates from smaller U.S. surveys.

The population quantities measured by ACS and the smaller survey

must be related, but are not assumed to be the same.

The bivariate modeling is very simple!

No covariates from auxiliary information are used!

We illustrate huge reductions in variances for our examples!

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 2 / 15
Introduction

We investigate the potential for borrowing strength from ACS

estimates via bivariate small area estimation models to reduce

variances of estimates from smaller U.S. surveys.

The population quantities measured by ACS and the smaller survey

must be related, but are not assumed to be the same.

The bivariate modeling is very simple!

No covariates from auxiliary information are used!

We illustrate huge reductions in variances for our examples!

Note: Examples use data prior to the COVID-19 pandemic.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 2 / 15
Three U.S. surveys

American Community Survey (ACS)

Samples approximately 3.5 million addresses each year.

Many topics: demographics, income, health insurance, housing,

disabilities, occupations, employment, education, etc

Produces estimates based on 1 or 5 years of data.

National Health Interview Survey (NHIS)

About 97,000 persons in sample for 2016 Early Release (ER) estimates.

Questions about a broad range of health topics through personal

household interviews.

Survey of Income and Program Participation (SIPP) Disability

Module

Approximately 37,000 households and 70,000 persons in 2008 panel.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 3 / 15
Three applications

1 NHIS estimates of U.S. state uninsured rates.

ACS variable: Previous 1-year estimates of U.S. state uninsured rates

(timing, questions asked, and the mode of survey delivery and design

di§er from NHIS).

In a CNSTAT workshop report, Kenney and Lynch (2010) noted a

consensus that ìthe NHIS produces the most valid [insurance] coverage

estimates,î but also that ìthe [NHIS] sample size is too small to

produce precise annual state and substate estimates for most statesî

2 SIPP estimates of U.S. state disability rates.

ACS variable: 1-year estimate of state disability rates

2008 SIPP panel asked more detailed questions about disability than

did ACS.

3 ACS 1-year county estimates of poverty rates of school-aged

children

2nd variable: Previous ACS 5-year estimates ñ these have a larger

sample size, but are less current.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 4 / 15
Bivariate Gaussian model

y1i = Y1i + e1i = (μ1 + u1i) + e1i, i = 1, . . . , m.

y2i = Y2i + e2i = (μ2 + u2i) + e2i



u1i

u2i



i.i.d  N(0, Σ), Σ =



σ11 σ12

σ12 σ22



e1i

e2i



ind N(0, Vi), Vi =



vi,11 0

0 vi,22

y1iis the direct estimate of the population characteristic from the

smaller survey, and y2iis the related ACS direct estimate.

Naive approach: include y2i as a covariate in the model for Y1i, but

this ignores its sampling error! See Bell, Chung, Datta, Franco (2019)

Parameterize model via μ1, μ2, σ11, σ22 and ρ = corr(u1i, u2i)

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 5 / 15
Prediction when model parameters are known

In matrix notation yi = (Yi) + ei = (μ + ui) + ei

Y^

BP

i = E(Yi

jyi) = μ + Σ(Σ + Vi)

1

(yi  μ)

MSE(Y^BP

i

) = Var(Yijyi) = Σ  Σ(Σ + Vi)

1Σ

We are interested in predicting Y1i only, not Y2i

In what follows, all models are given a hierarchical Bayes

treatment (using JAGS) with di§use priors on the parameters.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 6 / 15
Application 1: NHIS estimates of U.S. state uninsured

rates

y1i = 2016 NHIS estimate, y2i = 2015 ACS estimate

Smoothing of NHIS direct sampling variances is applied

Only 45 direct NHIS state estimates were published due to

ìconsiderations of sample size and precision.î

●

●

● ● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

4 6 8 10 12 14 16

0

5 10 15

PCT UNINSURED ESTIMATES

ACS DIRECT

NHIS DIRECT

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

● ● ●

●

● ● ●

●

0.0 0.5 1.0 1.5 2.0 2.5 3.0

0 20 40 60 80

VAR % DIFF BIV & UNIV vs DIR

NHIS VARIANCE

VARIANCE PCT DIFF

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 7 / 15
Application 1: NHIS variance decreases from modeling

^ρ = E(ρjfy1i g, fy2i g) = .97

percentage variance reductions

model mean 1st q. median 3rd q. max

univariate Gaussian 11 7 11 15 19

bivariate Gaussian 62 53 66 72 78

Table: Percent variance reductions from direct estimates for the univariate and

bivariate models

The results may suggest that it might be possible to publish

estimates for more states using a bivariate model, due to the

substantially lowered variances.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 8 / 15
Application 2: 2010 SIPP state total disability estimates

y1i = SIPP estimate, y2i = ACS estimate

Smoothing of SIPP direct variances is applied

^ρ = .96

percentage variance reductions

model mean 1st q. median 3rd q. max

univariate Gaussian 22 8 20 32 66

bivariate Gaussian 41 21 39 57 85

Table: Percent variance reductions from direct estimates for the univariate and

bivariate models.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 9 / 15
Application 3: ACS 1-year estimates of county poverty

rates for school-age children

2012 county rates of children in poverty used as illustration (good

regressors are available, but excluded here).

y1i = 2012 ACS 1-year estimate., y2i = 2007-2011 ACS 5-year

estimate

ρˆ = 0.94

percentage variance reductions

model mean 1st q. median 3rd q. 95 p.

univariate Gaussian 33 17 32 47 65

bivariate Gaussian 62 54 67 74 81

Table: Percent variance reductions from direct estimates for the univariate and

bivariate models

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 10 / 15
Alternative bivariate models

Because our applications involve proportions, we also Öt univariate

and bivariate versions of two other models

Binomial logit normal model (Franco and Bell 2015):

Binomial assumption for sampling model using e§ective sample size

and e§ective number of successes

logit transformation of true proportions for linking model

Unmatched sampling and linking model (You and Rao 2002):

Gaussian assumption for sampling model

logit tansformation of true proportions for linking model

These alternatives show large % reductions of variances similar,

overall, to the Gaussian model.

Point predictions (posterior means) are similar accross models, but

posterior variances di§er.

Franco and Bell (2021) propose a model selection criterion, but the

results are not deÖnitive. More research is needed.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 11 / 15
Concluding remarks

Large variance decreases are possible by using bivariate models to

borrow strength from ACS estimates to improve estimates from

smaller U.S. surveys, provided ρ is high!

Given the wide range of population characteristics estimated by ACS,

many smaller U.S. surveys estimate some closely related (or ostensibly

the same) characteristics for which this borrowing of information could

achieve substantial variance reductions.

Generally little or no beneÖts can be realized by trying to borrow

information from a smaller survey to improve the estimates from a

much larger survey.

Bivariate model is simple, easy to apply

Further research is needed on model comparison statistics

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 12 / 15
Disclaimers

All U.S. Census Bureau disclosure avoidance guidelines have been followed

and estimates have been approved for release by the U.S. Census Bureau

Disclosure Review Board. DRB approval number: CBDRB-FY19-357.

The views expressed in this presentation are those of the authors and not

of the U.S. Census Bureau or the National Opinion Research Center.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 13 / 15
References

Bell, William R., Chung, Hee-Cheol, Datta, Gauri S., and Franco, Carolina

(2019), ìMeasurement Error in Small Area Estimation: Functional Versus

Structural Versus Naive Models,î Survey Methodology, 45, 61ñ80

Franco, Carolina and Bell, William R. (2015), ìBorrowing Information Over

Time in Binomial/Logit Normal Models for Small Area Estimation,î

Statistics in Transition (new series) and Survey Methodology, joint issue on

Small Area Estimation, 16, 563ñ584, available at

http://stat.gov.pl/en/sit-en/issues-and-articles-sit/previous-issues/volume16-number-4-december-2015/

Franco, Carolina and Bell, William R. (2021). ìUsing American Community

Survey Data to Improve Estimates from Smaller U. S. Surveys through

Bivariate Small Area Estimation Modelsî Journal of Survey Statistics and

Methodology, to appear.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 14 / 15
References

Kenney, Genevieve and Lynch, Victoria (2010). Monitoring Childrenís Health

Insurance Coverage Under CHIPRA Using Federal Surveys, chapter 8 in

Databases for Estimating Health Insurance Coverage for Children: A

Workshop Summary. Thomas J. Plewes, rapporteur. Committee on National

Statistics, Division of Behavioral and Social Sciences and Education.

Washington, DC: The National Academies Press.

You, Yong and Rao, J. N. K. (2002), ìSmall Area Estimation Using

Unmatched Sampling and Linking Models,îThe Canadian Journal of

Statistics, 30, 3ñ15.

Bell & Franco () Combining ACS and smaller survey data November 3, 2021 15 / 15