Geographic Area Prevalence Estimates for Food Insecurity

Katherine Li1, Yajuan Si12, Brady T. West2, John A. Kirlin3, and

Xingyou Zhang4

1Dept. of Biostatistics, University of Michigan—Ann Arbor

2

Institute for Social Research

3Kirlin Analytic Services

4Bureau of Labor Statistics

November 3, 2021

The findings and conclusions in this presentation are those of the authors and

should not be construed to represent any official USDA or U.S. Government

determination or policy.

1 / 19
Motivation

Small area estimation (SAE) estimates the population quantity of

interest (e.g. food insecurity prevalence) for an “area” (e.g.

demographic groups or geographic tract) that has sparse data where

direct estimates could be unstable.

Developing techniques that can improve the SAE precision is crucial

for data-driven decision making in health and policy research.

Model-based SAE approaches fit models to survey data and use

external data from auxiliary or population records for prediction.

However, accounting for multi-stage complex sampling design features,

such as strata, clusters, and weights of the survey data, poses a

challenge for generalizable estimates.

2 / 19
Motivation

Goal: Produce valid prevalence estimates of food insecurity for tracts,

counties, and states (“small areas”) in the contiguous U.S. with a sample

survey

Setting: The survey was obtained under a complex sampling design and

did not include observations from all areas of interest.

We consider two approaches: weighted analysis and synthetic

population generation analysis.

3 / 19
Outline

1 Data

2 Methods

Weighted Analysis

Synthetic Population Generation Analysis

3 Results

4 / 19
Datasets

National Household Food Acquisition and Purchase Survey

(FoodAPS): Collected person-level data (n=9,894) from 4,826

non-institutionalized households April 2012–January 2013.

I Multi-stage stratified cluster sampling design. Sampling weights

calibrated to totals in the 2013 Current Population Survey (CPS).

I Included pseudo-strata and pseudo-PSUs for variance estimation.

I Collected variables expected to be predictive of food security.

Enriched sample: FoodAPS linked with ACS 2010 and SNAP 2012

(n=9,894). Used to fit the outcome model.

External Prediction dataset: ACS 2010 linked with SNAP 2012

(N=7,982,966). Used with estimated model coefficients to make

out-of-sample predictions.

5 / 19
General point estimation procedure

Let the outcome be yj = 1 if person j is food-insecure; yj = 0 otherwise.

1 Fit the outcome model using the enriched sample to obtain

coefficient estimates βˆ.

logit(P(yj = 1|Xj)) = Xjβ

2 Use predictors Xg,pred from prediction dataset to make person-level

predictions (yˆ

g = Xg,predβ, ˆ g = 1 . . . , N)

3 Aggregate relevant yˆ

g

into small area estimates θˆarea, where area is

the tract, county, or state of interest.

Note that in step (1), we model the data with weighted logistic regression

for the weighted analysis and unweighted logistic regression for the

synthetic population generation analysis.

6 / 19
General point estimation workflow

7 / 19
Methods: Overview

8 / 19
Weighted analysis: weighted logit with jackknife SE

This approach separates point and variance estimation steps.

I Use weighted logistic regression to obtain point estimates

I Use jackknife replication to obtain variance estimates.

9 / 19
Weighted analysis

Let h index the pseudo-strata; i index the pseudo-PSUs; ch refer to the

number of pseudo-PSUs in stratum h; and {whij} reference the base

weights from the enriched sample.

1 Point estimation:

I Employ the point estimation procedure, using the base weights {whij}

in the weighted logistic regression.

2 Replicate weight construction:

I A set of replicate weights {w

∗

hij}(hi)

is constructed by dropping

observations in the ith PSU and renormalizing weights in stratum h

such that P

i∈h w

∗

hij =

P

i∈h whij

3 Variance estimation via jackknife replication:

1 Repeat the point estimation procedure for each set of replicate weights

using {w

∗

hij}(hi)

in the weighted regression

2 Obtain {θ

ˆarea

(hi)

}

3 var(θ

ˆarea) = PH

h=1

ch

ch−1

Pch

i=1

(θ

ˆarea

(hi) − θ

ˆarea)

2

10 / 19
Workflow for weighted analysis

11 / 19
Synthetic Population Generation Analysis

This approach integrates point and variance estimation steps.

The Weighted Finite Population Bayesian Bootstrap (WFPBB)

“undoes” the complex sampling design by drawing unobserved units

from the weighted Pólya distribution

These “synthetic populations” can then be analyzed as a SRS without

sampling uncertainty.

12 / 19
Synthetic Population Generation Analysis

1 WFPBB replicate weights for bootstrap sample Bl

:

1 For each stratum h, take a SRSWR of ch − 1 PSUs. The number of

resamples is m∗

ih for the ith PSU.

2 Construct the replicate weights {w

∗

hij} such that

w

∗

hij = ch/(ch − 1)m∗

ihwhij and Pw

∗

hij =

Pwhij .

2 WFPBB synthetic population generation for Bl

:

I Use {w

∗

hij} in the weighted Pólya to draw enough units s.t. after

combining with Bl, synthetic population Sl,fis of size Pn

j=1 whij .

I Employ the point estimation procedure using the unweighted logistic

regression on each Sl,f, → θˆarea

l,f

I The estimate for Bl

is θ

ˆarea

l =

1

F

PF

f

θ

ˆarea

l,f

3 Point estimation:

I θˆarea =

1

L

PL

l

θˆarea

l

4 Variance estimation:

I V(θ

ˆarea) = (1 + 1/L)

1

L−1

PL

l

(θ

ˆarea

l − θ

ˆarea)

2

13 / 19
Workflow for WFPBB

14 / 19
Results

We report point and total variance estimates for food insecurity

prevalence at the U.S. and state levels from the weighted and

synthetic population generation analyses.

Results are compared to external estimates from the December 2012

CPS Food Security Supplement.

95% confidence intervals are constructed based on the total SE

(

q

sampling SE2 + model SE2)

15 / 19
Results: US overall food insecurity prevalence

Estimate (%) Total variance 95% CI

Direct estimate 28.2 0.2 (27.3, 29.1)

Weighted estimate 15.7 1.0 (13.6, 17.7)

Weighted analysis 16.4 0.4 (15.0, 17.7)

Synth. pop. analysis 16.5 1.3 (14.2, 18.9)

CPS estimate 14.5 – –

16 / 19
Results: State-level point estimates

17 / 19
Results: State-level sampling SE

18 / 19
Discussion

WFPBB estimates higher sampling SE than jackknife because we

discard bootstrap samples that do not have all levels of the categorical

variables used in the model. SE estimates are much closer to that of

jackknife when the model does not include the problematic variables

(division, urban/rural status).

Confidence intervals of the synthetic population analysis cover more of

the CPS state-level estimates in out-of-sample areas.

A substantial portion of tract-level variation is left unaccounted for,

and there are no additional predictive tract-level variables in ACS that

we can include in the model.

Next steps:

I Re-structure model so there are fewer sparse variables

I Use embedded MRP (multilevel regression and poststratification)

methods which will allow the model to include FoodAPS variables not

found in the prediction data.

19 / 19