Fitting a Bayesian Fay-Herriot Model

Nathan B. Cruze

United States Department of Agriculture

National Agricultural Statistics Service (NASS)

Research and Development Division

Washington, DC

October 25, 2018

“. . . providing timely, accurate, and useful statistics in service to U.S. agriculture.” 1
Disclaimer

The Findings and Conclusions in This Preliminary Presentation Have Not Been Formally Disseminated by the U.S.

Department of Agriculture and Should Not Be Construed to

Represent Any Agency Determination or Policy.
Overview

◮ NASS interest in small area estimation (SAE)

◮ The Fay and Herriot (1979) model

◮ Case study: county estimates of planted corn, Illinois 2014

◮ Computation in R and JAGS

GASP 2018–Fitting Bayesian Fay-Herriot Model 3
Small Area Estimation (SAE) Literature

“A domain is regarded as ‘small’ if the domain-specific sample is

not large enough to support [survey] estimates of adequate

precision.”–Rao and Molina (2015)

Regression and mixed-modeling approaches in SAE literature

◮ Shrinkage–improve estimates with other information

◮ Utility of auxiliary data as covariate

◮ Variance-bias trade off

Two common models

1. Unit-level models, e.g., Battese et al. (1988)

◮ USDA NASS (formerly SRS) as source of data/funding

2. Area-level models, e.g., Fay and Herriot (1979)

GASP 2018–Fitting Bayesian Fay-Herriot Model 4
NASS Interest In SAE

Iwig (1996): USDA’s involvement in county estimates in 1917

Published estimates used by:

◮ Agricultural sector

◮ Financial institutions

◮ Research institutions

◮ Government and USDA

Published estimates used for:

◮ County loan rates

◮ Crop insurance

◮ County-level revenue

guarantee

National Academies of Sciences, Engineering, and Medicine (2017)

◮ Consensus estimates: Board review of survey and other data

◮ Currently published without measures of uncertainty

◮ Recommends transition to system of model-based estimates

GASP 2018–Fitting Bayesian Fay-Herriot Model 5
Fay-Herriot (Area-Level) Model

Fay and Herriot (1979)–improved upon per capita income

estimates with following model

θ

ˆ

j = θj + ej

, j = 1, . . . , m counties (1)

θj = 1

′

j β + uj (2)

Adding Eqs. 1 and 2

θˆ

j = 1

′

j β + uj + ej

◮ θˆ

j

, direct estimate

◮ E(ej

|θj) = 0

◮ V(ej

|θj) = ˆσ

2

j

, estimated

variance

◮ 1j

, known covariates

◮ uj

, area random effect

◮ uj

iid∼ (0, σ2

u

)

GASP 2018–Fitting Bayesian Fay-Herriot Model 6
Fay-Herriot Formulated As Bayesian Hierarchical Model

‘Recipe’ for hierarchical Bayesian model as in Cressie and Wikle

(2011)

Data model:

θˆ

j

|θj, β

ind∼ N(θj

, σˆ

2

j

) (3)

Process model:

θj|β, σ2

u

iid∼ N(1

′

j β, σ2

u

) (4)

Prior distributions on β and σ

2

u

◮ Browne and Draper (2006), Gelman (2006): σ

2

u ∼?

◮ We will specify σ

2

u ∼ Unif (0, 108

), β

iid∼ MVN(0, 106

■)

Goal: Obtain posterior summaries about county totals, θj

GASP 2018–Fitting Bayesian Fay-Herriot Model 7
County Agricultural Production Survey (CAPS)

Case study in Cruze et al. (2016)

Illinois planted corn

◮ 9 Ag. Statistics Districts

◮ 102 counties

◮ a major producer of corn

◮ End-of-season survey

– Direct estimates of totals

– Estimated sampling variances

Min Median Max

n reports 2 47 93

CV (%) 9.1 19.2 92.3

Warren 

187 

Hende son 

071 

McDonough 

109 

Stephenson 

1n 

Carroll 

015 

Ogle 

141 

Le 

Winnebago Boon 

201 007 

McHenry 

111 

DeKalb Kane 

037 089 

Lake 

097 

Cook 

10 103 

- �� ---1 Kendall Will 

197 

Knox 

095 

Fulton 

057 

Bureau 

011 

LaSalle 

099 20 

093 

Grundy 

063 

Peoria 

143 

Macoupin 

117 

Kankake 

091 

Livingston 

- - --. 105 Iroquois 

Woodford 075 

203 

Montgom 

135 

McLean 

113 50 Ford 

053 40 _ ___ , Vermilion 

_� ,- Champaign 183 

DeWitt 

039 

Macon 

115 

019 

Coles 

029 

Edgar 

045 

Clark 70 Cumberland 023 . -- -� - I _�

035 --�- -� 

Fayete 

051 

Effingham 

049 

Jasper 

079 

. --.,Marion 

Clay 

025 

. ----- 121 

Jefferson 

081 

Franklin 

055 

Williamson 

199 

https://www.nass.usda.gov/Charts_and_Maps/Crops_

County/indexpdf.php
Covariate x1: USDA Farm Service Agency (FSA) Acreage

◮ FSA administers farm

support programs

◮ Enrollment popular,

not compulsory

◮ Data self-reported at

FSA office

◮ Administrative vs.

physical county

https://www.fsa.usda.gov/news-room/efoia/electronic-reading-room/

frequently-requested-information/crop-acreage-data/index
Covariate x2: NOAA Climate Division March Precipitation

Weather as auxiliary variable

◮ March: Planting ‘intentions’

◮ April: Illinois planting

◮ Could rainfall in March

affect planting?

◮ One-to-one mapping: ASD

and climate division

◮ Repeat value for all counties

within ASD

ASD Precip (in)

10 1.08

20 1.35

30 1.27

40 1.66

50 1.50

60 1.36

70 1.46

80 1.69

90 2.00

Source: ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv

Details in Vose et al. (2014)

GASP 2018–Fitting Bayesian Fay-Herriot Model 10
NASS Official Statistics

From prior publication: Illinois 2014, 11.9 million acres of corn

planted

◮ Require: State-ASD-county benchmarking of estimates

State/district: https://quickstats.nass.usda.gov/results/3A17F375-B762-37BD-8C03-D581DC8F7A85

County: https://quickstats.nass.usda.gov/results/478D1A7B-E680-3E5E-95E4-9A59F938A256
JAGS Model

◮ Note data, process, prior structure from earlier slide

◮ Note distributions parameterized in terms of precision

◮ Read into R script as stored R source code or as text string

Online resources http://www.sumsar.net/blog/2013/06/three-ways-to-run-bayesian-models-in-r/
A Pseudo-Code R Script

GASP 2018–Fitting Bayesian Fay-Herriot Model 13
Analysis of JAGS Model Output

Posterior summaries of parameters–based on 3,000 saved iterates

◮ Posterior means, standard deviations, quantiles, potential

scale reduction factors, effective sample sizes, pD, DIC

◮ Transform back to acreage scale

◮ Ratio benchmarking–inject benchmarking factor back into

chains as in Erciulescu et al. (2018)

GASP 2018–Fitting Bayesian Fay-Herriot Model 14
Results: Models With and Without Benchmarking

◮ Modeled estimates (ME) may not satisfy benchmarking

◮ Ratio-benchmarked estimates (MERB) are consistent with

state targets and improve agreement with external sources

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

County Comparisons of Model and FSA Acreage

FSA Planted Area (Acres of Corn)

Modeled Estimates of Planted Area (Acres)

● MERB estimate

ME estimate

●

●

●

●

●

●

●

●

●

ASD Comparisons of Model and FSA Acreage

FSA Planted Area (Acres of Corn)

Modeled Estimates of Planted Area (Acres)

● MERB estimate

ME estimate

GASP 2018–Fitting Bayesian Fay-Herriot Model 15
Results: Posterior Distributions of ASD-Level Acreages

Used county-level inputs to produce county-level estimates

◮ Idea: derive ASD-level estimates from Monte Carlo iterates

◮ Sum corresponding draws from county posterior distributions

– Compute means and variances from aggregated chains

500 1000 1500 2000

Planted Area (1,000 Acres)

ASD 80 ASD 90

ASD 20 ASD 30

ASD 70

ASD 40

ASD 50

ASD 60

ASD 10

MERB

NASS OFFICIAL

GASP 2018–Fitting Bayesian Fay-Herriot Model 16
Results: Relative Variability of Survey Versus Model

Obtain estimates and measures of uncertainty for counties and

districts

◮ Recall the goal of SAE–increased precision!

CV (%) of CAPS Survey Estimates

Min Q1 Median Mean Q3 Max

County 9.1 16.6 19.2 22.2 23.5 92.3

District 4.4 5.6 6.8 6.6 7.2 8.7

CV (%) of MERB Estimates

Min Q1 Median Mean Q3 Max

County 3.6 5.6 7.2 9.0 10.5 31.2

District 1.7 2.0 2.1 2.5 2.3 4.4

GASP 2018–Fitting Bayesian Fay-Herriot Model 17
Results: Comparison to Other Sources

For counties and districts, compute ‘standard score’

◮ (model estimate-other source)/model standard error

◮ Direct Estimates, Cropland Data Layer, Battese-Fuller, FSA

● ● ●

● ● ●

● ●●

● ●

−6 −4 −2 0 2 4 6

County−Level Comparisons

Number of Posterior Standard Deviations

FSA Battese_Fuller CDL DE*

●●

FSA Battese_Fuller CDL DE

−2 0 2 4 6

ASD−Level Comparisons

Number of Posterior Standard Deviations

GASP 2018–Fitting Bayesian Fay-Herriot Model 18
Conclusions

Discussed Bayesian formulation of Fay-Herriot model motivated by

NASS applications

Other R packages facilitate Bayesian small area estimation

◮ ‘BayesSAE’ by Chengchun Shi

◮ ‘hbsae’ by Harm Jan Boonstra

◮ May be bound by limited choice of prior distributions

◮ Transformations of data may be needed

Proc MCMC in SAS added ‘Random’ statement as of version 9.3

Thanks to Andreea Erciulescu (NISS) and Balgobin Nandram

(WPI) for three years of adventures in small area estimation!

GASP 2018–Fitting Bayesian Fay-Herriot Model 19
References

Battese, G. E., Harter, R. M., and Fuller, W. A. (1988). An error-components model for prediction of county crop

areas using survey and satellite data. Journal of the American Statistical Association, 83(401):28–36.

Browne, W. J. and Draper, D. (2006). A comparison of bayesian and likelihood-based methods for fitting multilevel

models. Bayesian Analysis, 1(3):473–514.

Cressie, N. and Wikle, C. (2011). Statistics for Spatio-Temporal Data. Wiley, Hoboken, NJ.

Cruze, N., Erciulescu, A., Nandram, B., Barboza, W., and Young, L. (2016). Developments in Model-Based

Estimation of County-Level Agricultural Estimates. In Proceedings of the Fifth International Congress on

Establishment Surveys. American Statistical Association, Geneva.

Erciulescu, A. L., Cruze, N. B., and Nandram, B. (2018). Model-based county level crop estimates incorporating

auxiliary sources of information. Journal of the Royal Statistical Society: Series A (Statistics in Society).

doi:10.1111/rssa.12390.

Fay, R. E. and Herriot, R. A. (1979). Estimates of income for small places: An application of james-stein

procedures to census data. Journal of the American Statistical Association, 74(366):269–277.

Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models (comment on article by

browne and draper). Bayesian Analysis, 1(3):515–534.

Iwig, W. (1996). The National Agricultural Statistics Service County Estimates Program. In Schaible, W., editor,

Indirect Estimators in U.S. Federal Programs, chapter 7, pages 129–144. Springer, New York.

National Academies of Sciences, Engineering, and Medicine (2017). Improving Crop Estimates by Integrating

Multiple Data Sources. The National Academies Press, Washington, DC.

Rao, J. and Molina, I. (2015). Small Area Estimation. In Wiley Online Library: Books. Wiley, 2nd edition.

Vose, R. S., Applequist, S., Squires, M., Durre, I., Menne, M. J., Williams, C. N., Fenimore, C., Gleason, K., and

Arndt, D. (2014). Improved Historical Temperature and Precipitation Time Series for U.S. Climate Divisions.

Journal of Applied Meteorology and Climatology, 53(5):1232–1251.

GASP 2018–Fitting Bayesian Fay-Herriot Model 20