Improving the Utility of Poisson-Distributed, Differentially

Private Synthetic Data via Prior Predictive Truncation with an

Application to CDC WONDER

Harrison Quick (Drexel University)

Funding for this research came from the National Science Foundation (NSF-SES-1943730)
Table of Contents

Motivating Example: CDC WONDER

Methods for Generating Differentially Private Synthetic Data

Multinomial-Dirichlet Model

Poisson-Gamma Model

Prior Predictive Truncated Poisson-Gamma Model

Analysis of the PA Cancer Death Data

Summary
Table of Contents

Motivating Example: CDC WONDER

Methods for Generating Differentially Private Synthetic Data

Multinomial-Dirichlet Model

Poisson-Gamma Model

Prior Predictive Truncated Poisson-Gamma Model

Analysis of the PA Cancer Death Data

Summary
CDC WONDER
CDC WONDER
CDC WONDER

For the sake of illustration, we’ll look at county-level data for ages 35–44 from 2016
CDC WONDER

These ICD-10 codes represent “death due to heart disease”
CDC WONDER

County-level heart disease-related death counts for ages 35–44 in 2016 from all races

and all genders

All counts less than 10 are suppressed in public-use datasets
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Privacy

Is there a way that CDC can address these issues?
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Urban/Rural disparities

◮ Privacy

Is there a way that CDC can address these issues?
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Urban/Rural disparities

◮ Racial disparities

◮ Privacy

Is there a way that CDC can address these issues?
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Urban/Rural disparities

◮ Racial disparities

◮ Differences by sex

◮ Privacy

Is there a way that CDC can address these issues?
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Urban/Rural disparities

◮ Racial disparities

◮ Differences by sex

◮ Differences by age

◮ Privacy

Is there a way that CDC can address these issues?
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Urban/Rural disparities

◮ Racial disparities

◮ Differences by sex

◮ Differences by age

◮ Differences by cause-of-death

◮ Privacy

Is there a way that CDC can address these issues?
CDC WONDER

While CDC WONDER offers a wealth of data and does implement privacy protections,

there is still room for improvement:

◮ Utility: Suppression of small counts affects users’ ability to assess...

◮ Urban/Rural disparities

◮ Racial disparities

◮ Differences by sex

◮ Differences by age

◮ Differences by cause-of-death

◮ Privacy

◮ Targeted attacks by clever intruders can overcome data suppression to uncover the

true counts

Is there a way that CDC can address these issues?
Synthetic Data

One option to address the issue of data suppression would be to release synthetic data:

e.g., if

◮ y = (y1, . . . , yI)

T

denotes a restricted-use dataset,

◮ p (y | φ) is an appropriate statistical model for y with parameters φ, and

◮ p (φ | ψ) is a prior distribution for φ given hyperparameters, ψ,

then we can generate a synthetic dataset, z = (z1, . . . , zI)

T

, from the posterior

predictive distribution,

p (z | y, ψ) = Zp (z | φ) p (φ | y, ψ) dφ.

More specifically, we can sample φ

∗

from p (φ | y, ψ) and then sample z from p (z | φ

∗

).
Differentially Private Synthetic Data(Dwork, 2006)

The standard typically used for demonstrating formal privacy guarantees is the concept

of differential privacy (Dwork, 2006).

In this context, p (z | y, ψ) is ǫ-differentially private if for any similar1 dataset, x,









log p (z | y, ψ)

p (z | x, ψ)









≤ ǫ. (1)

While ψ can be viewed as a vector of model parameters — selected a priori in hopes

that E [yi| ψ] ≈ yiin order to produce synthetic data with high utility — the elements

of ψ are primarily used to satisfy ǫ-differential privacy.

1

kx − yk = 2 and P

i

xi =

P

i

yi — i.e., there exists i and i

′

such that xi = yi − 1 and xi

′ = yi

′ + 1

with all other values equal
Table of Contents

Motivating Example: CDC WONDER

Methods for Generating Differentially Private Synthetic Data

Multinomial-Dirichlet Model

Poisson-Gamma Model

Prior Predictive Truncated Poisson-Gamma Model

Analysis of the PA Cancer Death Data

Summary
Multinomial-Dirichlet model

Let y be a vector of sensitive count data of length I ≥ 2 with P

i

yi = y· and assume

y | θ ∼ Mult (y·, θ) and θ ∼ Dir (α).

It can (but won’t) be shown that if

min αi ≥ z·/ [exp (ǫ) − 1] ,

the multinomial-Dirichlet synthesizer, p (z | y, α), will satisfy ǫ-differential privacy.

◮ i.e., if our Dir (α) prior is informative enough, it can sufficiently mask the data.

Key drawback: Assumes homogeneity

◮ Shouldn’t Philadelphia have more deaths than Small Town, PA?

◮ Shouldn’t more deaths be attributed to old people than young people?
Poisson-Gamma model

In contrast, the Poisson-gamma framework assumes

yi| λi ∼ Pois (niλi) and λi ∼ Gamma (ai, bi).

Since the yi are (conditionally) independent Poisson random variables, we can write

y |λ,

X

i

yi = y· ∼ Mult y·,

(

P

niλi

j

njλj

)!

◮ Allows for heterogeneity in population sizes (via ni) and underlying event rates

(via ai/bi)

But under what conditions will this satisfy ǫ-differential privacy?
Poisson-Gamma model

It can (but won’t) be shown that the Poisson-gamma synthesizer, denoted

p (z | y, a, b), will satisfy ǫ-differential privacy if

ai ≥

z·

e

ǫ/νi − 1

(2)

where νi ∈ [1, 2] denotes what amounts to a penalty term associated with the

additional information gained from using the Poisson-gamma model compared to the

multinomial-Dirichlet model.

Key drawback: Extreme “worst case scenario”

◮ Above criteria protects against group with ONE observed event (yi = 1) being

assigned ALL of the synthetic events (zi = z·).

◮ e.g., all cancer-related deaths in PA being assigned to a single rural county — this

shouldn’t be possible, so why should we worry about this???
Prior predictive truncated Poisson-gamma framework

Rather than focus on technical details, let’s consider a hypothetical example.

Suppose E [yi| a, b, n] = niλi0 = 10 for a given i and that our dataset consists of

y· > 26,000 events. Then 99.9% of the prior predictive distribution falls between

yi = 2 and yi = 22.

> qpois(.0005,10)

[1] 2

> qpois(.9995,10)

[1] 22

◮ If y· > 26,000, we should expect a reduction in model

informativeness on the order of

26,000

22 − 2

> 1,300

Note: This approach is heavily dependent on having high quality prior information

◮ If E [yi

| a, b, n] = niλi0 6≈ yi, then the prior predictive bounds will not be good.

◮ We will need to rely on subject-matter experts to know what is sufficient
Table of Contents

Motivating Example: CDC WONDER

Methods for Generating Differentially Private Synthetic Data

Multinomial-Dirichlet Model

Poisson-Gamma Model

Prior Predictive Truncated Poisson-Gamma Model

Analysis of the PA Cancer Death Data

Summary
PA cancer death data from 1980 — 26,116 deaths from 47,034 strata

Attribute Levels

County i = 1, . . . , 67 Counties in Pennsylvania

Cancer Type

c = 1, . . . , 9 Forms of Cancer

Cancers of the lip, oral cavity, and pharynx (ICD-9: 140–149);

Cancers of the digestive organs and peritoneum (ICD-9: 150–159);

Cancers of the respiratory and intrathoracic organs (ICD-9: 160–165)

Cancers of the breast (ICD-9: 174–175);

Cancers of the genital organs (ICD-9: 179–187);

Cancers of the urinary organs (ICD-9: 188–189);

Cancers of all other and unspecified sites (ICD-9: 170–173, 190–199);

Leukemia (ICD-9: 204–208);

and all other cancers of the lymphatic and hematopoietic tissues (ICD-9: 200–203)

Age

a = 1, . . . , 13 Levels

Ages under 1; Ages 1–4; Ages 5–9; Ages 10–14; Ages 15–19; Ages 20–24; Ages 25–34;

Ages 35–44; Ages 45–54; Ages 55–64; Ages 65–74; Ages 75–84; and Ages 85 and older

Race r = 1, . . . , 3 Levels (Black, White, and Other)

Sex s = 1, 2 Levels (Male and Female)

Overview of the structure of the Pennsylvania cancer data. Cancer types are identified by their International Classification of Diseases, Ninth Revision

(ICD-9) codes. Data are publicly available — free of suppression — because they predate the privacy protections.
Prior information: National death rates vs. PA death rates

Years of Age

Deaths per 100,000

0 20 40 60 80

0 0.1 1 10 1000

Lip

Digestive

Lung

Other/Unspec.

Breast

Genital

Urinary

Other Lymph.

Leukemia

US Average

PA Average

Cause-specific death rates at the national level and for the state of Pennsylvania.

National-level rates are used as prior information for estimating the proper allocation of

deaths at the state and county level.

◮ We don’t need these to be perfect, we just need them to comparable.
Utility of synthetic data: Age-adjusted rates

140 180 220 260

140 180 220 260

True Rates

Synthetic Rates

ε=1

ε=4

(a) Original/Untruncated

140 180 220 260

140 180 220 260

True Rates

Synthetic Rates

ε=1

ε=4

(b) Prior Predictive Truncation

Figure 1: Comparison of age-adjusted cancer-related death rates based on the two approaches

for generating synthetic data for ǫ = 1 and ǫ = 4.

◮ When ǫ = 1, the original model requires all ai > 15,000, whereas the prior

predictive truncation approach has a max (ai) < 17 and most are less than 0.58
Utility of synthetic data: Age-adjusted rates

(a) True Age-Adjusted Rates (b) Synthetic Age-Adjusted Rates

Death Rate

(per 100,000)

Below 193

193 − 200

200 − 207

207 − 214

214 − 221

Over 221

◮ Because the prior information does not account for geographic disparities, as

ǫ → 0, estimates become geographically homogenous

◮ Point of emphasis: More difficult to identify true disparities, but also unlikely to

produce spurious disparities

ε = 4
Utility of synthetic data: Urban/rural and black/white disparities

0 1 2 3 4

1.00 1.10

Privacy Budget (ε)

Urban/Rural Disparity

Truth

Prior

(a) Urban/Rural Disparity

0 1 2 3 4

1.0 1.2 1.4 1.6 1.8

Privacy Budget (ε)

Black/White Disparity

Men

Women

Truth

Prior

(b) Black/White Disparity

◮ Here again, as ǫ → 0, our estimates of the disparities shift from “the truth” to

“the prior”.

◮ Prior information did not include anything about urban/rural disparities, so the

effect is attenuated toward the null (i.e., no disparity)
Utility of synthetic data: Why we need to release the prior information

0 1 2 3 4

1.0 1.2 1.4 1.6

Privacy Budget (ε)

Black/White Disparity

Truth

Prior

(a) Digestive Cancer; Females

0 1 2 3 4

0.4 0.8 1.2 1.6

Privacy Budget (ε)

Black/White Disparity

Truth

Prior

(b) Other Lymphatic Cancer; Males

◮ Digestive Cancer; Females: National black/white disparity is larger than in PA

◮ Other Lymphatic Cancer; Males: National black/white disparity is the opposite of

the disparity in PA

◮ Disclosing the prior information will help users determine if the results from the

synthetic data are driven by the data or are a reflection of the prior
Table of Contents

Motivating Example: CDC WONDER

Methods for Generating Differentially Private Synthetic Data

Multinomial-Dirichlet Model

Poisson-Gamma Model

Prior Predictive Truncated Poisson-Gamma Model

Analysis of the PA Cancer Death Data

Summary
Summary

Using the prior predictive distribution to truncate the range of values for the

Poisson-gamma model can reduce the model’s informativeness by several orders of

magnitude, thereby producing substantial increases in utility

◮ The utility of the prior predictive truncation approach is heavily reliant on the

quality of the prior information; e.g., mortality rates differ by age, thus our prior

information ought to differ by age

◮ A small amount of our privacy budget can be used to protect the prior

information. Ideally, the prior would be based on relatively large counts (e.g.,

national death counts) such that adding DP noise would be unlikely to cause any

meaningful changes.
Thanks for listening!

References:

◮ Machanavajjhala, A., Kifer, D., Abowd, J., Gehrke, J., and Vilhuber, L. (2008).

“Privacy: Theory meets practice on the map.” In IEEE 24th International

Conference on Data Engineering, 277–286.

◮ Quick, H. (2021). “Generating Poisson-distributed differentially private synthetic

data.” J. Roy. Statist. Soc., Ser. A (Statistics in Society), 184, 1093–1108.

◮ Quick, H. “Improving the utility of Poisson-distributed, differentially private

synthetic data via prior predictive truncation with an application to CDC

WONDER.” arXiv preprint 2103.03833.