Estimating Preferences Over Data to Inform

Statistical Disclosure Methods Decisions

Elan Segarra

U.S. Bureau of Labor Statistics

FCSM

October 25th, 2023

Disclaimer: The views expressed herein are those of the author(s) and do not

necessarily reflect those of the Federal Government, Department of Labor, or the

Bureau of Labor Statistics. All results have been reviewed to ensure that no

confidential information is disclosed.

0 / 21
BLS Publishes Many Many Statistics

1 / 21
Many Dimensions of Publication Choice

How we implement statistical disclosure control (SDC) methods is

one important dimension:

1. When using cell suppression approaches, which

complementary cells should we suppress?

2. When using formally private methods, how should we divvy up

the privacy budget among published statistics?

2 / 21
Many Dimensions of Publication Choice

How we implement statistical disclosure control (SDC) methods is

one important dimension:

1. When using cell suppression approaches, which

complementary cells should we suppress?

2. When using formally private methods, how should we divvy up

the privacy budget among published statistics?

To help, we need a way to measure relative value of publications...

2 / 21
Project Overview

Goal: Quantify public data users’ preferences over the statistics

that BLS publishes.

Data:

• Census of Fatal Occupational Injuries (CFOI): Data being

consumed

• Google Analytics: Data used to estimate value/preferences

Method: Estimate a nested logit model of consumer preferences.

Results:

• CFOI stats broken down by employment status have the

highest value

• CFOI preference estimates are not useful for informing cell

suppression disclosure control methods

• However they are useful for allotting privacy budgets in

formally private disclosure control methods (e.g. differentially

private noise infusion)

3 / 21
Subject of Interest: CFOI

The Census of Fatal Occupational Injuries (CFOI) is the subject

data set for this project:

• Annual census of fatal work injuries collected since 1992

• Compiled by a Federal-State cooperative program which

collects data info multiple sources (police reports, news,

OSHA investigations, etc.)

• Compiled data includes narratives, injury codes (OIICS),

geography, timing, and demographic information.

4 / 21
Subject of Interest: CFOI

The Census of Fatal Occupational Injuries (CFOI) is the subject

data set for this project:

• Annual census of fatal work injuries collected since 1992

• Compiled by a Federal-State cooperative program which

collects data info multiple sources (police reports, news,

OSHA investigations, etc.)

• Compiled data includes narratives, injury codes (OIICS),

geography, timing, and demographic information.

Disclosure control is particularly difficult for CFOI:

• It is a census, the counts are small, and some data is public

• BLS publishes many tables/figures/statistics using CFOI (e.g.

industry and occupational breakdowns)

• Currently use cell suppression to protect confidentiality

4 / 21
Model of Consumer Choice Over Statistics

There are three levels to the model of data consumer preferences:

1. A statistic is an individual number

• Ex: The count of work fatalities in the construction sector

(NAICS 23)

2. A publication is a collection of statistics

• Ex: A table/figure of work fatality counts by industry (2 digit

NAICS)

3. A market consists of a set of publications at a specific time

from which a data consumer can choose

• Ex: On BLS.gov a data consumer can choose to view fatality

counts by employee status, industry, occupation, or age group

Key insight: Observing which publications are chosen (i.e. clicked

on) reveals preferences over the underlying statistics

5 / 21
Model of Consumer Choice: Nested Logit

Consumer i has indirect utility from publication p in market t

given by

Uipt =

1

|Sp|

X

s∈Sp

Xstβ + Wptθ + ξpt

| {z }

≡δpt

+ςig + (1 − σ)εipt

Sp : Set of statistics included in publication p

Xst : Observable characteristics of statistic s (eg ind. breakdown)

Wpt : Observable characteristics of publication p (eg bar chart)

ξjt : Unobservable stat. characteristics (eg ugly presentation)

ςig : Unobservable correlated nest shock (eg broken site link)

εipt : Unobservable characteristics (eg researcher vs layperson)

Objects of interest: β and θ quantify preferences across

characteristics

6 / 21 Nest Structure
Model of Consumer Choice: Utility Maximization

If users choose the data product (e.g. table) that maximizes their

indirect utility, then the observed “market share” of publication p

in time period t is given by

spt =

exp 

δpt

1−σ



Dσ

g

P

h D

1−σ

h

where Dg =

X

k∈g

exp 

δkt

1 − σ



Note: “Market share” = spt =

qpt

Mt

=

pageviews

site visitors

7 / 21
Model of Consumer Choice: Utility Maximization

If users choose the data product (e.g. table) that maximizes their

indirect utility, then the observed “market share” of publication p

in time period t is given by

spt =

exp 

δpt

1−σ



Dσ

g

P

h D

1−σ

h

where Dg =

X

k∈g

exp 

δkt

1 − σ



Note: “Market share” = spt =

qpt

Mt

=

pageviews

site visitors

Estimation:

1. Inversion step (the “magic” of logit):

ln spt − ln s0t = X ̃

ptβ + Wptθ + σ ln sp|g + ξpt

2. Estimate using traditional regression techniques (e.g. Two

Stage Least Squares)

7 / 21
Data: Google Analytics

Google Analytics

• Granular data on pageviews, duration, and even demographics

• There are 28 different tables/figures each published over

multiple reference years

• Relative pageviews function as our measure of choice among

consumers

8 / 21
Data: Extracting Characteristics

Characteristics are manually coded for each site, such as

• ind = includes breakdown by industry sectors

• format = bar chart, table, time series etc.

• multiyear = includes more than one year of data

• curr year = includes most recent RY (at view time)

• exposition = includes exposition along with data

⇒

xpt =















curr year

ind

gender

event

.

.

.















=















1

0

1

1

.

.

.















wpt =











multiyear

format bar

exposition

.

.

.











=











0

1

0

.

.

.











9 / 21
Estimation Results: Statistic Characteristics

Though the exact values of βb are difficult to interpret, their

relative ordering reflects which breakdowns are more valued by

data consumers.

10 / 21 Full Regression Results
Estimation Results: Publication Characteristics

Similarly, the exact values of θb are difficult to interpret, but they

suggest which publication chars. are valued by data consumers.

11 / 21 Full Regression Results
Statistical Disclosure Control (SDC)

Consider these 2 tables of synthetic CFOI statistics:

Table 1: Fatalities by Age and Year

Year

Age 2019 2020 2021 Total

< 20 2 3 8 13

20-34 29 27 34 90

35-54 51 46 55 152

≥ 55 49 43 57 149

Total 131 119 154 404

Table 2: Fatalities by Age and Industry

Industry (for 2021)

Age Cons. Mfg. Trade Total

< 20 4 3 1 8

20-34 15 9 10 34

35-54 29 15 11 55

≥ 55 31 12 14 57

Total 79 39 36 154

How can we use our estimated preferences over data to inform our

SDC methods?

We consider two application cases:

1. Cell Suppression Problem

2. Formally Private Noise Injection

12 / 21
Using the Estimation Results

Theoretically, we can use the estimated preference parameters to

construct average valuations of any relevant statistics, e.g.:

• Table of fatalities by age and year: U1 = −2.75

• Table of fatalities by age and industry: U2 = −2.18

Since U1 < U2 we can conclude the second table is generally more

valuable than the first on average.

13 / 21
Using the Estimation Results

Theoretically, we can use the estimated preference parameters to

construct average valuations of any relevant statistics, e.g.:

• Table of fatalities by age and year: U1 = −2.75

• Table of fatalities by age and industry: U2 = −2.18

Since U1 < U2 we can conclude the second table is generally more

valuable than the first on average.

There are two issues:

1. Negative utility/valuations may be inappropriate (e.g. for

dividing a privacy budget)

2. Difficult to ascribe relative valuation since utility is equivalent

up to positive affine transformation.

13 / 21
Using the Estimation Results

Approach:

• Identify a set of statistics and push them through the nested

logit demand model

• Yields choice probabilities that can be used like valuations

• They are all positive and between 0 and 1 so can function

nicely in disclosure control methods

Example

Given those 2 table options then the model predicts:

• P(Choose Table 1) = s1 = 0.361

• P(Choose Table 2) = s2 = 0.639

This estimates that the publication of fatalities by age and industry

is approximately 77% more valuable (on average) as a publication

that breaks down by age and industry.

14 / 21
Disclosure Application: Cell Suppression Problem

SDC Method: Tabular Cell Suppression

• Sensitive cells are suppressed to protect confidentiality.

• Additional cells (i.e. complementary/secondary) often need to

be suppressed.

Industry (for 2021)

Age Cons. Mfg. Trade Total

< 20 4 3 1 8

20-34 15 9 10 34

35-54 29 15 11 55

≥ 55 31 12 14 57

Total 79 39 36 154

Potential

Set of

Complementary

Suppressions

Primary

Suppression

15 / 21
Disclosure Application: Cell Suppression Problem

SDC Method: Tabular Cell Suppression

• Sensitive cells are suppressed to protect confidentiality.

• Additional cells (i.e. complementary/secondary) often need to

be suppressed.

Industry (for 2021)

Age Cons. Mfg. Trade Total

< 20 4 3 1 8

20-34 15 9 10 34

35-54 29 15 11 55

≥ 55 31 12 14 57

Total 79 39 36 154

Potential

Set of

Complementary

Suppressions

Primary

Suppression

15 / 21
Disclosure Application: Cell Suppression Problem

SDC Method: Tabular Cell Suppression

• Sensitive cells are suppressed to protect confidentiality.

• Additional cells (i.e. complementary/secondary) often need to

be suppressed.

Industry (for 2021)

Age Cons. Mfg. Trade Total

< 20 4 3 1 8

20-34 15 9 10 34

35-54 29 15 11 55

≥ 55 31 12 14 57

Total 79 39 36 154

Potential

Set of

Complementary

Suppressions

Primary

Suppression

There are often multiple options for complementary suppressions.

Estimated valuations over these cells can help guide decisions.

15 / 21
Disclosure Application: Cell Suppression Problem

Table 1: Fatalities by Age and Year

Year

Age 2019 2020 2021 Total

< 20 2 3 8 13

20-34 29 27 34 90

35-54 51 46 55 152

≥ 55 49 43 57 149

Total 131 119 154 404

Value

5%

2.5%

0%

Table 2: Fatalities by Age and Industry

Industry (for 2021)

Age Cons. Mfg. Trade Total

< 20 4 3 1 8

20-34 15 9 10 34

35-54 29 15 11 55

≥ 55 31 12 14 57

Total 79 39 36 154

We can then use these individual cell valuations as an input into

any CSP solver to find optimal complementary suppressions.

16 / 21
Disclosure Application: Cell Suppression Problem

Table 1: Fatalities by Age and Year

Year

Age 2019 2020 2021 Total

< 20 2 3 8 13

20-34 29 27 34 90

35-54 51 46 55 152

≥ 55 49 43 57 149

Total 131 119 154 404

Value

5%

2.5%

0%

Table 2: Fatalities by Age and Industry

Industry (for 2021)

Age Cons. Mfg. Trade Total

< 20 4 3 1 8

20-34 15 9 10 34

35-54 29 15 11 55

≥ 55 31 12 14 57

Total 79 39 36 154

We can then use these individual cell valuations as an input into

any CSP solver to find optimal complementary suppressions.

Note: Doesn’t perform markedly different from optimizing the

number of suppressed cells because:

1. Estimated cell valuations do not have much variation

2. GA data don’t allow for disentangling many cell valuations

16 / 21
Disclosure Application: Differentially Private Tables

Differential Privacy (DP):

• Publication property that provides a provable confidentiality

guarantee

• Increasing adoption of DP across the FSS

• Mechanisms involve noise injection (e.g. perturbing cells)

• Typically involve a privacy budget, e.g. ε

• Larger ε ⇒ less noise and less security

17 / 21
Disclosure Application: Differentially Private Tables

Differential Privacy (DP):

• Publication property that provides a provable confidentiality

guarantee

• Increasing adoption of DP across the FSS

• Mechanisms involve noise injection (e.g. perturbing cells)

• Typically involve a privacy budget, e.g. ε

• Larger ε ⇒ less noise and less security

How to use estimated valuations:

• Allocate the privacy budget, ε, among publications, e.g.

• For Table 1 and 2 use 0.361ε and 0.639ε.

• More generally, for P potential publications

1. Estimate valuations: sb1, . . . ,sbP

2. For publication p use sbpε in the DP mechanism

• Maintains publication accuracy where most valued by data

consumers

17 / 21
Conclusion

Summary:

• Presented a proof of concept of how to use estimated

preferences over statistics to inform SDC methods

• Developed a model of consumer choice over publications and

estimated it for CFOI using Google Analytics data

• Found significant heterogeneity in preferences over publication

characteristics

• Though these estimated preferences are not pivotal for solving

cell suppression problem, they are useful for allotting privacy

budgets in formally private SDC methods

Next Steps:

• Expand the model to the random coefficients case (i.e. BLP)

to allow for the inclusion of consumer demographics

• Explore the framework with other BLS data products

18 / 21
CONTACT INFORMATION

Thank You!

Elan Segarra

U.S. Bureau of Labor Statistics

Office of Compensation and Working Conditions

Segarra.Elan@BLS.gov

19 / 21
Nested Logit Model: Decision Tree

The nests (i.e. groups of related statistics/tables) line up with the

likely route a user takes to access the statistic or table:

Choose Data Portal

IIF

Factsheets

Interactive

Tables

State

Tables

DB

Tool

Confined

Spaces

Workplace

Violence Cranes Emp.Status Ind. Occup. AL AK . . . WY

Can possibly generalize to other access mediums (e.g. twitter or

API) or even to entire catalog of BLS data products.

20 / 21 ——Back——
Estimation

To put all of this in a more familiar form, if we define

yjt = ln qjt − ln q0t

Then we have

yjt = Xjtβ + σ ln sj|g + ξjt

and since yjt, Xjt, and sj|g are all observed we can use traditional

regression methods to estimate β and σ.

20 / 21
Estimation

To put all of this in a more familiar form, if we define

yjt = ln qjt − ln q0t

Then we have

yjt = Xjtβ + σ ln sj|g + ξjt

and since yjt, Xjt, and sj|g are all observed we can use traditional

regression methods to estimate β and σ.

Open question: Can we use OLS or is there endogeneity that

would require something like IV?

• Reminder: ξjt are unobs. table characteristics

• Seems like obs. table characteristics (Xjt) are exogenously

determined by BLS

20 / 21
Regression Results (Full)

(1) (2)

Employment Status 0.518∗∗ 0.518∗∗

(0.037) (0.038)

Industry -0.112∗∗ -0.112∗∗

(0.030) (0.033)

Occupation -0.459∗∗ -0.459∗∗

(0.034) (0.036)

Gender 0.069 0.069

(0.072) (0.058)

Injury Event -0.453∗∗ -0.453∗∗

(0.025) (0.029)

Age -0.525∗∗ -0.525∗∗

(0.042) (0.038)

Geo. State -0.350∗∗ -0.350∗∗

(0.088) (0.107)

Race/Ethnicity -0.509∗∗ -0.509∗∗

(0.034) (0.034)

Document Source -0.958∗∗ -0.958∗∗

(0.054) (0.061)

Constant -3.308∗∗ -3.308∗∗

(0.182) (0.118)

** indicates significance at the 0.01 level.

(1) (2)

Type: Count -0.016 -0.016

(0.030) (0.028)

Type: Percent -0.482∗∗ -0.482∗∗

(0.075) (0.062)

Type: Rate -0.179∗∗ -0.179∗∗

(0.036) (0.034)

Form: Bar Chart 0.337∗∗ 0.337∗∗

(0.051) (0.060)

Form: Line Chart 0.660∗∗ 0.660∗∗

(0.058) (0.051)

Multiple Ref. Years -0.274∗∗ -0.274∗∗

(0.021) (0.025)

Inc. Current Ref. Year 1.170∗∗ 1.170∗∗

(0.059) (0.080)

Nest Shares 0.358∗∗ 0.358∗∗

(0.029) (0.037)

Mkt FE Yes Yes

Robust SE No Yes

N 5870 5870

** indicates significance at the 0.01 level.

21 / 21 ——Back——