1/ 26

Private Table Statistics using Synthetic

Microdata Generation

Terrance D. Savitsky 1 Matthew R. Williams 2

Monika (Jingchen) Hu 3

1 U.S. Bureau of Labor Statistics (Office of Survey Methods Research)

2RTI International (Division for Statistical and Data Sciences)

3Vassar College (Mathematics and Statistics Department)

FCSM 2023 Conference

October 25, 2023
2/ 26

Overview

I Goal: Generate private data for tabular release

I e.g. Tables of counts and salaries

I point estimates and SE estimates

I Approach: Synthesize data with privacy guarantee

I Model both outcome y and survey weights w

I Two ways:

I 1. Model under observed sample distribution

I 2. Model under population distribution

I Results:

I Compare synthesizers with additive noise mechanism
3/ 26

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
4/ 26

Differential privacy

I D ∈ R

n×k

be a database in input space D

I Mechanism M() : R

n×k → O.

I M is -differentially private if

P r[M(D) ∈ O]

P r[M(D

0

) ∈ O]

≤ exp(),

I Probability M(D

0

) assigns to O changes by max of exp() after deleting

1 row

I For all D, D

0

∈ D that differ by 1 row.
5/ 26

M = Additive Noise

I An output statistic f(D); e.g., total employment

I Global sensitivity

∆G = supD,D0∈D: δ(D,D0

)=1 | f(D) − f(D

0

) |

I Laplace Mechanism for additive noise, scaled to be

proportional to ∆G/ with −DP guarantee

I Survey weights dramatically increase ∆G.

I Adding noise disrupts tabular constraints
6/ 26

M = Pseudo posterior distribution

ξ

α(y)

(θ | y) ∝

Yn

i=1

p(yi| θ)

αi × ξ(θ)

I Down weight each likelihood by αi ∈ [0, 1]

I αi

lower when disclosure risk higher

I Sensitivity of ξ

α(y)

(θ | y) → f

α

θ

(y) = log Qn

i=1 p(yi

| θ)

αi

.

I ∆α = sup

y,y0∈Yn:δ(y,y0)=1

sup

θ∈Θ

|α(y) × fθ(y) − α(y

0

) × fθ(y

0

)|

I Posterior draw with y = 2∆α produces one synthetic y

∗

I y

∗ produces survey tables with same privacy guarantee
7/ 26

Synthesizers Encode Privacy by Smoothing

1e+01

1e+03

1e+05

Data Synthesizer

Distribution (log)
8/ 26

Asymptotic DP for M = Pseudo Posterior

To justify a global DP result (bounding all data sets) compared

to a local DP result (bound observed data set):

Asymptotic “Discovery” of ∆α at large sample sizes (n)

I Space of plausible values Θ collapses to a point θ

∗

, so

don’t need to look at supθ∈Θ.

I Variation across local ∆α,x collapses onto ∆α.

I Achieves (, δ)− pDP, where δ > 0 is probability ∃x ∈ X n

exceeding the  bound.

I δ → 0 at O(n

−1/2

).

I Requires increasing sparsity in downweighted record

contributions, which aligns with focus on isolated records

as risky.
9/ 26

Data from Survey sampling procedure

I Sample S of size n taken from population U of size N

I Each individual in U assigned selection probability

P(ωi = 1 | A) = πi

I Estimate area statistics with survey weights wi = 1/πi

, to

reduce bias

I Survey weights, (wi), designed to correct bias

I Incorporating privacy designed to induce bias
10/ 26

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
11/ 26

Two Synthesizing Models

I Synthesis of a local survey database (yn, wn|Xn, αn):

I A Fully Bayes model for observed sample (FBS) models

(yn, wn|Xn, αn) under a multinormal pseudo likelihood.

I A Fully Bayes model for the population (FBP) that forms

the exact likelihood for (yn|Xn, αn),(wn|yn, Xn, αn) in the

observed sample.

(yi, wi|xi, αi, ωi = 1) = P r(ωi = 1|yi

, xi, wi) × (yi, wi|xi, αi)

P r(ωi = 1|xi, wi)

I FBP produces synthesized y

∗

n without sampling bias, so

discard weights to build tabular statistics

I FBS requires use of both (y

∗

n

, w∗

n

).
12/ 26

Estimation Algorithm

1. Estimate unweighted θ with model,

ξ (θ|y, w) ∝ [

Qn

i=1 π (yi

, wi|θ)] × π (θ)

2. Compute weights,

αi = m (supθ∈Θ|fθ (yi, wi)|) ∝ 1/ supθ∈Θ|fθ (yi, wi)|

3. Re-estimate θ using weights, αiin

ξ

α (θ | y, wγ) ∝ [

Qn

i=1 π (yi

, wi| θ)

αi

] π (θ | γ)

4. Compute log-likelihood bound,

supyi,wi∈Dn supθ∈Θ|α(yi

, wi)fθ(yi, wi)| ≤ ∆α

5. Gives us privacy guarantee,  ≤ 2∆α

6. Generate synthetic data, (y

∗

, w∗) ∼ πα (y

∗

, w∗|y, w)
13/ 26

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
14/ 26

Computing Sensitivity

I The local sensitivity ∆c

f,g for count of field f and gender g

(cell count):

∆c

f,g = max

i∈Sf,g

wi − min

i∈Sf,g

wi

I The local sensitivity ∆a

f,g for average salary of field f and

gender g (cell average):

∆a

f,g =

maxi∈Sf,g wiyi − mini∈Sf,g wiyi

P

i∈Sf,g

wi − (maxi∈Sf,g wi − mini∈Sf,g wi)

I ∆c

∗ = maxf,g ∆c

f,g and ∆a

∗ = maxf,g ∆a

f,g

I Generate noise Laplace(0, ∆f,g

c,a

∗ /) added to cell count

and average salary
15/ 26

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
16/ 26

SDR application (10,355 obs): model fits

Figure: Distributions of record-level Lipschitz bounds of the

non-private unweighted and the private weighted of FBS (left) and

FBP (right) in the SDR application.
17/ 26

SDR application (10,355 obs): utility evaluation

Figure: RMSE values of counts (left) and average salary values

(right) of the three methods, FBS, FBP, and Laplace, applied to the

SDR sample. Each violin plot represents a distribution of RMSE

values over 27 cells. Results are based on m = 3 synthetic datasets

by FBS and FBP, achieving yn = 10.8 for all three methods.
18/ 26

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
19/ 26

Simulation studies: simulation design

I Based on the 2017 SDR public use file

I Population N = 100, 000 units of: salary (yi), field of

expertise and gender (xi)

I Salary yi

| xi ∼ Lognormal(μi, 0.4) where μiis

group-specific mean from the public use file

I Additive noise: noisei ∼ Lognormal(0, 0.4)

I Survey weights:

log(πi) = log(yi) + noisei

wi = 1/πi

I Take a stratified PPS sample of n = 1000 using fields as

strata: (yn, Xn, wn)
20/ 26

Sensitivity before/after weighting by α

Figure: Distributions of record-level sensitivity bounds of the

non-private unweighted and the private weighted of FBS (left) and

FBP (right) in the simulation.
21/ 26

Smoothed weights E(w|y) Improves Efficiency

Figure: Comparison of the salary and weight bivariate distributions of

confidential salary and weights in the sample (green and left),

synthetic salary and smoothed weights from FBS (yellow and middle),

and synthetic salary and smoothed weights from FBP (blue and right).
22/ 26

Utility of M = Synthesizers versus M = Laplace

Figure: RMSE ratios of counts (left) and average salary values

(right) of the three methods, FBS, FBP, and Laplace, applied to the

selected sample. Each violin plot represents a distribution of RMSE

ratios over 27 cells. Results are based on m = 3 synthetic datasets

by FBS and FBP, achieving yn = 10.8 for all three methods.
23/ 26

Adding Synthetic Data Replicates, m, Improves Utility

Figure: RMSE ratios of counts (left) and average salary values

(right) of FBS and FBP, applied to the selected sample. A red dashed

line at RMSE ratio = 1 is included for reference. Each violin plot

represents a distribution of RMSE ratios over 27 cells. Results are

based on m = {1, 3, 5} synthetic datasets by FBS and FBP, achieving

yn = {3.6, 10.8, 18} for both methods.
24/ 26

Outline

Differential privacy

Two synthesizers

Laplace Mechanism

Survey of Doctoral Recipients Application

Simulation Study

Concluding remarks
25/ 26

Summary

I Formal privacy for data collected under an informative

sampling design

I We recommend the FBS: easy to estimate and produces

low RMSE

I The synthetic data is privacy protected and obeys all

constraints without any post processing

I No interactive queries required

I The synthetic data may be used for other purposes

I arXiv link to manuscript:

https://arxiv.org/abs/2101.06188
26/ 26

References

I Leon-Novelo, L. G. and Savitsky, T. D. (2019). Fully

Bayesian estimation under informative sampling, Electronic

Journal of Statistics, 13, 1608-1645.

I Savitsky, T. D., Williams, M. R. and Hu, J. (2020). Bayesian

pseudo posterior mechanism under differential privacy.

arXiv:1909.11796.

I Rao, J. N. K., Wu, C. F. J. and Yue, K. (1992). Some recent

work on resampling methods for complex surveys, Survey

Methodology, 18, 209-217.