1/ 24

Private Tabular Survey Data Products

through Synthetic Microdata Generation

Terrance D. Savitsky 1 Matthew R. Williams 2

Monika (Jingchen) Hu 3

1 U.S. Bureau of Labor Statistics (Office of Survey Methods Research)

2NSF (National Center for Science and Engineering Statistics)

3Vassar College (Mathematics and Statistics Department)

FCSM Conference

November, 2021
2/ 24

Overview

◮ Goal: Generate private data for tabular release

◮ e.g. Tables of counts and salaries

◮ point estimates and SE estimates

◮ Approach: Synthesize data with privacy guarantee

◮ Model both outcome y and survey weights w

◮ Two ways:

◮ 1. Model under observed sample distribution

◮ 2. Model under population distribution

◮ Results:

◮ Compare synthesizers with additive noise mechanism
3/ 24

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
4/ 24

Differential privacy

◮ D ∈ R

n×k

be a database in input space D

◮ Mechanism M() : R

n×k → O.

◮ M is ǫ-differentially private if

P r[M(D) ∈ O]

P r[M(D

′

) ∈ O]

≤ exp(ǫ),

◮ Probability M(D

′

) assigns to O changes by max of exp(ǫ) after deleting

1 row

◮ For all D, D

′

∈ D that differ by 1 row.
5/ 24

M = Additive Noise

◮ An output statistic f(D); e.g., total employment

◮ Global sensitivity

∆G = supD,D′∈D: δ(D,D′

)=1 | f(D) − f(D

′

) |

◮ Laplace Mechanism for additive noise, scaled to be

proportional to ∆G/ǫ with ǫ−DP guarantee

◮ Survey weights dramatically increase ∆G.

◮ Adding noise disrupts tabular constraints
6/ 24

M = Pseudo posterior distribution

ξ

α(y)

(θ | y) ∝

Yn

i=1

p(yi| θ)

αi × ξ(θ)

◮ Down weight each likelihood by αi ∈ [0, 1]

◮ αi

lower when disclosure risk higher

◮ Sensitivity of ξ

α(y)

(θ | y) based on

fθ(y) = log Qn

i=1 p(yi

| θ)

αi

.

◮ ∆α = supy,y′∈Yn:δ(y,y′)=1|α(y) × fθ(y) − α(y

′

) × fθ(y

′

)|

◮ Each posterior draw with ǫy = 2∆α produces one synthetic

y

∗

◮ Synthetic y

∗ produces survey tables with same privacy

guarantee
7/ 24

Data from Survey sampling procedure

◮ A sample S of n individuals taken from population U of size

N

◮ Each individual in U assigned selection probability

P(ωi = 1 | A) = πi

◮ Estimate area statistics with survey weights wi = 1/πi

, to

reduce bias

◮ Survey w designed to correct bias

◮ Incorporating privacy leads to distortion (bias)
8/ 24

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
9/ 24

Two Synthesizing Models

◮ Synthesis of a local survey database (yn, wn|Xn, αn):

◮ A Fully Bayes model for observed sample (FBS) models

(yn, wn|Xn, αn) under a multinormal pseudo likelihood.

◮ A Fully Bayes model for the population (FBP) that forms

the exact likelihood for (yn|Xn, αn),(wn|yn, Xn, αn) in the

observed sample.

(yi, wi|xi, αi, ωi = 1) = P r(ωi = 1|yi

, xi, wi) × (yi, wi|xi, αi)

P r(ωi = 1|xi, wi)

◮ FBS produces synthesized y

∗

n without sampling bias, so

discard weights to build tabular statistics

◮ FBS requires use of both (y

∗

n

, w∗

n

).
10/ 24

Estimation Algorithm

1. Estimate unweighted θ with model,

ξ (θ|y, w) ∝ [

Qn

i=1 π (yi

, wi|θ)] × π (θ)

2. Compute weights,

αi = m (supθ∈Θ fθ (yi, wi)) ∝ 1/ supθ∈Θ fθ (yi, wi)

3. Re-estimate θ using weights, αiin

ξ

α (θ | y, wγ) ∝ [

Qn

i=1 π (yi

, wi| θ)

αi

] π (θ | γ)

4. Compute log-likelihood bound,

supyi,wi∈Dn supθ∈Θ|α(yi

, wi)fθ(yi, wi)| ≤ ∆α

5. Gives us privacy guarantee, ǫ ≤ 2∆α

6. Generate synthetic data, (y

∗

, w∗) ∼ πα (y

∗

, w∗|y, w)
11/ 24

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
12/ 24

Computing Sensitivity

◮ The local sensitivity ∆c

f,g for count of field f and gender g

(cell count):

∆c

f,g = max

i∈Sf,g

wi − min

i∈Sf,g

wi

◮ The local sensitivity ∆a

f,g for average salary of field f and

gender g (cell average):

∆a

f,g =

maxi∈Sf,g wiyi − mini∈Sf,g wiyi

P

i∈Sf,g

wi − (maxi∈Sf,g wi − mini∈Sf,g wi)

◮ ∆c

∗ = maxf,g ∆c

f,g and ∆a

∗ = maxf,g ∆a

f,g

◮ Generate noise Laplace(0, ∆f,g

c,a

∗ /ǫ) added to cell count

and average salary
13/ 24

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
14/ 24

SDR application (10,355 obs): model fits

Figure: Distributions of record-level Lipschitz bounds of the

non-private unweighted and the private weighted of FBS (left) and

FBP (right) in the SDR application.
15/ 24

SDR application (10,355 obs): utility evaluation

Figure: RMSE values of counts (left) and average salary values

(right) of the three methods, FBS, FBP, and Laplace, applied to the

SDR sample. Each violin plot represents a distribution of RMSE

values over 27 cells. Results are based on m = 3 synthetic datasets

by FBS and FBP, achieving ǫyn = 10.8 for all three methods.
16/ 24

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
17/ 24

Simulation studies: simulation design

◮ Based on the 2017 SDR public use file

◮ Population N = 100, 000 units of: salary (yi), field of

expertise and gender (xi)

◮ Salary yi

| xi ∼ Lognormal(μi, 0.4) where μiis

group-specific mean from the public use file

◮ Additive noise: noisei ∼ Lognormal(0, 0.4)

◮ Survey weights:

log(πi) = log(yi) + noisei

wi = 1/πi

◮ Take a stratified PPS sample of n = 1000 using fields as

strata: (yn, Xn, wn)
18/ 24

Sensitivity before/after weighting by α

Figure: Distributions of record-level sensitivity bounds of the

non-private unweighted and the private weighted of FBS (left) and

FBP (right) in the simulation.
19/ 24

Smoothed weights E(w|y) Improves Efficiency

Figure: Comparison of the salary and weight bivariate distributions of

confidential salary and weights in the sample (green and left),

synthetic salary and smoothed weights from FBS (yellow and middle),

and synthetic salary and smoothed weights from FBP (blue and right).
20/ 24

Utility of M = Synthesizers versus M = Laplace

Figure: RMSE ratios of counts (left) and average salary values

(right) of the three methods, FBS, FBP, and Laplace, applied to the

selected sample. Each violin plot represents a distribution of RMSE

ratios over 27 cells. Results are based on m = 3 synthetic datasets

by FBS and FBP, achieving ǫyn = 10.8 for all three methods.
21/ 24

Adding Synthetic Data Replicates, m, Improves Utility

Figure: RMSE ratios of counts (left) and average salary values

(right) of FBS and FBP, applied to the selected sample. A red dashed

line at RMSE ratio = 1 is included for reference. Each violin plot

represents a distribution of RMSE ratios over 27 cells. Results are

based on m = {1, 3, 5} synthetic datasets by FBS and FBP, achieving

ǫyn = {3.6, 10.8, 18} for both methods.
22/ 24

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
23/ 24

Summary

◮ Formal privacy for data collected under an informative

sampling design

◮ We recommend the FBS: easy to estimate and produces

low RMSE

◮ The synthetic data is privacy protected and obeys all

constraints without any post processing

◮ No interactive queries required

◮ The synthetic data may be used for other purposes

◮ arXiv link to manuscript:

https://arxiv.org/abs/2101.06188
24/ 24

References

◮ Leon-Novelo, L. G. and Savitsky, T. D. (2019). Fully

Bayesian estimation under informative sampling, Electronic

Journal of Statistics, 13, 1608-1645.

◮ Savitsky, T. D., Williams, M. R. and Hu, J. (2020). Bayesian

pseudo posterior mechanism under differential privacy.

arXiv:1909.11796.

◮ Rao, J. N. K., Wu, C. F. J. and Yue, K. (1992). Some recent

work on resampling methods for complex surveys, Survey

Methodology, 18, 209-217.