Distributional Privacy for Data Sharing
Abstract
Data sharing between different parties has become an important engine powering modern research and development processes.
An important class of privacy concerns in data sharing regards the underlying distribution of data.
For example, the total traffic volume of data from a networking company reveals the scale of its business.
Unfortunately, existing privacy frameworks do not adequately address this class of concerns.
In this paper, we propose distributional privacy , a framework for analyzing and protecting these distributional privacy concerns in data sharing scenarios.
Distributional privacy is applicable in multiple data sharing settings, including synthetic data release.
Theoretically, we analyze the lower and upper bounds of privacy-distortion trade-offs.
Practically, we propose data release mechanisms for protecting distributional privacy concerns, and demonstrate that they achieve better privacy-distortion trade-offs than alternative privacy mechanisms on real-world datasets.
Introduction
Data sharing between parties plays a critical role in modern research and development.
Data holders typically either release processed data [18, 9, 5, 21] or train a generative model (e g., GANs [8], diffusion models [11, 19, 20]) on their original data and share synthetic data with other parties.
For example, network traces shared from customers to networking vendors enable vendors to debug and improve products [24, 1].
Medical data shared between hospitals [7, 21] enables them to develop new machine-learning-based diagnosis algorithms collaboratively [4].
More generally, data shared from researchers allow their research to be reproducible by others [15].
However, data sharing also raises privacy concerns.
While the most well-studied privacy concerns relate to leaking user data, an important other class of concerns relates to the underlying data distribution .
For example, a video analytics company that shares synthetic video session data may wish to hide the total or mean traffic volume, which could imply the company’s total revenue [17].
A cloud provider that shares synthetic cluster performance traces may not want to reveal the proportions of different server types that the cloud provider owns, which are regarded as business secrets [15].
Note that this information (total/mean traffic volume, proportions of server types) cannot be inferred from any single record, but is inherent to the overall data distribution (or the aggregate dataset).
Existing privacy metrics and privacy-preserving data sharing algorithms do not adequately address these distributional privacy concerns , even in synthetic data scenarios .
They either focus on protecting
[Figure]
the privacy of individual records in a database (e.g., differential privacy [6], anonymization [18]), or are not suitable for synthetic data scenarios (e.g., attribute privacy [26], maximal leakage [12], privacy funnel [16]).
See more discussions in “related work” below.
Motivated by these observations, we argue that a new privacy framework is needed for defining, analyzing, and protecting distributional privacy concerns in data sharing settings.
In this paper, we take a first step towards the goal.
At a high level, the proposed framework works as follows (detailed formulation in §2).
A data holder first chooses one or more secrets, which are mathematically defined as functions of the data holder’s data distribution.
For example, a video analytics company might choose the mean daily observed traffic as a secret quantity.
Then, the data holder obfuscates their data according to some mechanism and releases the output (Fig. 1).
Our framework quantifies the privacy of this mechanism by analyzing the probability that an attacker can infer the data holder’s true secret after observing the output.
To capture the utility of released data, we define the distortion of a mechanism as the worst-case distance (where the distance metric can be chosen by the data holder or data user) between the original and released data distributions.
Our goal is to design mechanisms that control tradeoffs between privacy and distortion.
Related work.
Many existing techniques for privacy-preserving data release (both in the synthetic data setting and beyond) typically focus on protecting the privacy of individual records in a database.
For instance, in a database of demographic information, these techniques might obfuscate whether a specific user’s record is present in the database.
Concretely, differential privacy [6] evaluates how much individual samples influence the final output of the algorithm.
Similarly, popular privacypreserving data sharing algorithms like sub-sampling [18], anonymization [18, 3], or more recently, differentially-private deep generative models [13, 7, 15, 24], focus on protecting sensitive information from individual samples.
These techniques are not designed to protect distributional attributes at all (in fact, they are designed to preserve them).
Other frameworks like attribute privacy [26], maximal leakage [12], and privacy funnel [16] are designed to obfuscate sensitive random variables or properties of an aggregate dataset.
They can be adapted to protect distributional privacy concerns.
However, they are not necessarily well-suited to a data sharing setting.
For instance, attribute privacy considers the setting of releasing a statistical query of the dataset (instead of a full synthetic dataset).
Using it to release datasets would cause a poor privacy-distortion trade-off (see § 5).
Maximal leakage considers the worst-case secrets in discrete space.
But in our case, the secret is predefined by the data holder, and they could be in a continuous range (e.g., mean traffic).
This seemingly small difference causes much of the analysis from [12] to not extend to our setting.
Privacy funnel [16] evaluates the mutual information between a mechanism’s output and input.
This is ill-suited for data sharing, as a higher mutual information between the released and original datasets does not necessarily imply that they are close in distribution distance.
For readers interested in more technical details about these works, please refer to App. A.
Contributions.
Our contributions are as follows.
• Formulation: We formalize distributional privacy concerns in data sharing applications with the notion of distributional privacy (§ 2).
We propose new privacy and distortion metrics tailored to data sharing applications, including synthetic data sharing (§ 2).
• Analysis (upper and lower bounds):
We derive fundamental limits (lower bounds) on the tradeoff between privacy and distortion for a specific choice of distortion distance metric.
As a case study, we propose mechanisms that achieve order-optimal privacy-distortion tradeoffs when the secret is the mean of the distribution.
• Practical evaluation: We give preliminary empirical results showing how to use distributional privacy to release a real dataset, and how to evaluate the corresponding distributional privacy metric.
We show that our proposed algorithms achieve better privacy-distortion trade-offs than baseline mechanisms that were designed to protect different privacy metrics (§ 5).
Our proposed mechanisms are compatible with existing data synthesis tools (e
g., based on deep generative models [7, 15, 24, 13, 25]) and can be applied as an add-on to provide distributional privacy.
Distributional Privacy: Formulation
Notation.
We denote random variables with uppercase letters, and their realizations with lowercase.
For a random variable X , we denote its probability density function (PDF), or, in the case of discrete random variables, its probability mass function (PMF), as f X , and its distribution measure as ω X .
If a random variable X is drawn from a parametric family (e.g., X is Gaussian with specified mean and covariance); the parameters will be denoted with a subscript of X , i.e., the above notations become
X θ , f X θ , ω
X θ respectively for parameters θ ∈ R q , where q denotes the dimension of the parameters.
In addition, we denote f X | Y as the conditional PDF/PMF of X given another random variable Y .
Original data.
Consider a data holder who possesses a dataset of n samples X
= { x 1 , . . .
, x n } , where for each i ∈ [ n ] , x i ∈ R p is drawn i.i.d.
from an underlying parametric distribution.
That is, x i ∼ ω
X θ , where θ ∈ R q is a realization of random q -dimensional parameter vector Θ , and ω Θ is the probability measure for Θ .
In this paper, we assume that the data holder knows θ ; our results and mechanisms generalize it to the case when the data holder only possesses the dataset X (see § 4).
For example, suppose the original data samples come from a Gaussian distribution.
We have θ =
( µ, σ ) , and X θ ∼ N ( µ, σ ) .
ω Θ (or f Θ ) describes the prior distribution over ( µ, σ ) .
For example, if we know a priori that the mean of the Gaussian is drawn from a uniform distribution betweenand 1, and σ is always 1, we could have f Θ ( µ, σ ) =
I ( µ ∈ [0 , 1]) · δ ( σ ) , where I ( · ) is the indicator function, and δ is the Dirac delta function.
In practice, the underlying distribution can be much more complicated than a Gaussian.
Distributional secrets to protect.
We assume the data holder wants to hide ℓ ∈ Z > 0 secrets from the original data distribution, expressed as a function g ( θ ) :
R q → R ℓ .
In the Gaussian example
X θ ∼ N ( µ, σ ) , suppose the random variable X θ represents the traffic volume experienced by an enterprise in a day.
The data holder may wish to hide the mean traffic per day, in which case g ( · ) would be the mean of the distribution, i.e., g ( µ, σ )
= µ .
In general, the secret can be any (vectorvalued) function that can be deterministically computed from θ .
In this paper, we present general results for one-dimensional secrets (i.e., ℓ = 1 ) and defer a discussion of higher-dimensional secrets to future work (see § 6).
Data release mechanism.
The data holder releases data by passing the private parameter θ through a data release mechanism M g .
That is, for a given θ , the data holder first draws internal randomness z ∼ ω Z , and then releases another distribution parameter θ ′ =
M g ( θ, z ) , where M g is a purely deterministic function, and ω Z is a fixed distribution from which z is sampled.
We denote the random variable of the released data as Y θ ′ .
Note that here we assume that both the input and output of M g are distribution parameters.
It is straightforward to generalize to the case when the input and/or output are datasets of samples (see § 4).
For example, in the Gaussian case discussed above, the data release mechanism can be
M g (( µ, σ ) , z ) =
( µ + z, σ ) where z ∼ N (0 , 1) .
I.e., this mechanism shifts the mean of the
Gaussian by a random amount drawn from a standard Gaussian distribution and keeps the variance.
Attacker strategy.
Based on the released parameter θ ′ , the attacker outputs ˆ g ( θ ′ ) as a guess of the initial secret g ( θ ) .
ˆ g can be either random or deterministic.
The attacker is assumed to know the data release mechanism M g and output θ ′
but not the realization of the data holder’s internal randomness z .
For instance, in the running Gaussian example, an attacker may choose ˆ g ( µ ′ , σ ′ )
= µ ′ .
When the data holder releases a dataset of samples instead of the parameter θ ′ , this formulation can be used to upper bound attacker’s performance as if the released dataset has unlimited amount of samples and therefore can estimate the distribution parameter correctly.
Privacy metric.
The data holder cares about leaking its secrets.
We define our privacy metric privacy Π ϵ as the attacker’s probability of guessing the secret(s) to within a tolerance ϵ , worst-case over all attackers ˆ g :
[Equation]
The probability is taken over the randomness of the original data distribution ( θ ∼ ω Θ ), the data release mechanism ( z ∼ ω Z ), and the attacker strategy ( ˆ g ).
Distortion metric.
The main goal of data sharing is to provide useful data; hence, we (and data holders and users) want to understand how much the released data distorts the original data.
We define the distortion ∆ of a mechanism as the worst-case distance between the original distribution and the released distribution:
[Equation]
where d is a general distance metric defined over distributions.
The choice of the distance metric depends on the type of data and potentially depends on the applications that data holders and users care about.
In this paper, we adopt Wasserstein-
α distance as the starting point and all the results hold for any α ≥ 1 .
Wasserstein distance is often used for evaluating data quality (e.g., [24, 15]) and as the distance metric in neural network design (e.g., WGAN [2]).
Note that the definition in Eq.
(2) can be extended to data release mechanisms that take datasets as inputs and/or outputs.
Objective.
To summarize, the data holder’s objective is to choose a data release mechanism that minimizes privacy Π ϵ subject to a constraint on distortion metric ∆ :
[Equation]
An alternative formulation of minimizing ∆ subject to a constraint on Π ϵ is analyzed in App.
F.
Lower Bounds: Privacy-Distortion Tradeoffs
In this section, we study lower bounds for Eq.
(9); that is, given a budget for distortion ∆ , what are fundamental limits on achievable privacy metric Π ϵ ?
We first present a lower bound that applies regardless of the prior distribution of data ω Θ and regardless of the secret function g .
We assume that the secret is scalar (i.e., ℓ = 1 ).
Theorem 1 (Lower bound of privacy-distortion tradeoff) .
Let D ( X θ 1 , X θ 2 ) ≜
12 d (cid:0) ω
X θ 1 ∥ ω
X θ 2 (cid:1) , where d ( ·∥· ) denotes Wasserstein-
α distance.
Further, let R ( X θ 1 , X θ 2 ) ≜ | g ( θ 1 ) − g ( θ 2 ) | .
Let γ ≜ inf θ 1 ,θ 2 ∈ Supp ( ω Θ ) D
( X θ 1 ,X θ 2 )
R
( X θ 1 ,X θ 2 ) .
For any T > 0 , when ∆ ≤ T , we have Π ϵ ≥ ⌈ T 2 γϵ ⌉ − 1 .
The proof is in App.
B. Note that we have not made the quantity inf θ 1 ,θ 2 ∈ Supp ( ω Θ ) D
( X θ 1 ,X θ 2 )
R
( X θ 1 ,X θ 2 ) in the lower bound explicit, as its closed-form expression depends on the secret function g .
We have derived lower bounds for several scalar secret functions, including the mean, quantile, and standard deviation of continuous distributions, and the PMF of a discrete distribution evaluated at a specific point.
Due to space constraints, we present only the result for the mean of a continuous distribution.
Corollary 1 (Privacy lower bound, secret = mean of a continuous distribution) .
Consider the secret function g ( θ )
= (cid:82) x xf X
θ ( x ) dx .
For any T > 0 , when ∆ ≤ T , we have Π ϵ ≥ ⌈
Tϵ ⌉ − 1 .
The proof is in App.
C.
This lower bound is tight when ϵ divides T , as we show in the next section.
Upper bound: Data Release Mechanism Design
To satisfy distributional privacy, the data release mechanism (and corresponding privacy-distortion tradeoffs) depend on the secret function and potentially on the original data distribution.
Our high-level vision is to build a library of mechanisms for different common secret functions and families of parametric distributions; the data holder would then choose the setting that most closely matches their own data.
We have preliminary mechanism designs and analysis for a number of secret functions (e.g., mean, α -quantile, PMF value) and data distributions (e.g., Gaussian, exponential,
Pareto, geometric).
Due to space constraints, we present here results when the secret function is the mean of a one-dimensional continuous distribution that can be parameterized with a location parameter.
That is, the distribution parameter vector θ can be written as ( u, v ) , where u ∈ R , v ∈ R q − 1 , and for any u ̸ = u ′ , f X u,v ( x ) =
f X u ′ ,v ( x − u
′ + u ) .
The prior over distribution parameters is f U,V
( a, b ) =
f U ( a ) · f V ( b ) , where f U ( a ) = 1 u − u
I ( a ∈ [ u, u )) .
In other words, the prior distribution of the location parameter is uniform and independent of other factors.
Examples include the Gaussian, Laplace, and uniform distributions, as well as shifted distributions (e.g., shifted exponential, shifted log-logistic).
Algorithm input : θ = ( u, v ) where u is the mean of X θ = X u,v , and v ∈ R q − 1 controls other properties of the distribution, lower bound u of the mean value u , quantization interval s .
1 u
[Equation]
· s ; 2 v ′ ← v ;
[Equation]
u
[Equation]
z
[Equation]
← ( u
[Equation]
v ′ ) .
[Equation]
$
[Equation]
𝑢
[Equation]
.5𝑠
Shifting data release mechanism.
The data release mechanism we design is shown in Alg.
s is a hyperparameter of the mechanism that divides ( u − u ) , i.e., there exists N ∈ N such that s = u − uN .
Fig.
2 shows an example when the original data distribution is
Gaussian, i.e., X θ ∼ N ( u, v ) , and u ∈ [ u, u ) .
Our mechanism “quantizes” the range of possible mean values into segments of length s .
It then shifts the mean of private distribution f X u,v to the midpoint of its corresponding segment, and releases the resulting distribution f Y u ′ ,v ′ .
This simple deterministic mechanism is able to achieve the optimal privacy-distortion tradeoff in some cases, as shown below.
Theorem 2 (Optimality of Alg. 1 ) .
When 2 ϵ divides s , the data release mechanism in Alg.
1 achieves the lower bound in Thm.
The proof is in App.
D. Let ˆ g ∗ be the attack strategy that ˆ g ∗ ( u ′ , v ′ )
=
u ′ , i.e., the attack guesses the mean of private distribution as the release mean.
We also show in the proof that under the data release mechanism Alg. 1, ˆ g ∗ is the optimal attacker strategy (i.e., ˆ g ∗ ∈ arg sup
[Equation]
Extending the data release mechanism for handling dataset input/output.
Suppose the data holder possesses a dataset of i.i.d.
samples X = { x 1 , . . .
, x n } , where x i ∼ ω X u,v , i ∈ [ n ] .
The data release mechanism can be trivially extended to the dataset by shifting individual samples by the same amount as the distribution shift in Alg. 1:
[Equation]
As before, s is a hyperparameter of the mechanism that divides ( u − u ) .
In practice, since we do not know the true mean u , we estimate it by ˆ u = 1 n (cid:80) i ∈
[ n ] x i .
Note that this mechanism applies to samples.
Therefore, it can be applied either to the original data, or as an add-on to existing data sharing tools [7, 15, 24, 13, 25].
For example, it can be used to modify synthetically-generated samples after they are generated, or to modify the training dataset for a generative model, or to directly modify the original data for releasing.
Experiments
In this section, we experimentally compare our mechanism for achieving distributional privacy with privacy mechanisms that were designed to satisfy other privacy notions (e.g., differential privacy).
We demonstrate that prior mechanisms incur poor privacy-distortion tradeoffs under our metrics.
Dataset.
We use Wikipedia Web Traffic Dataset [9] for the experiment.
This dataset contains the daily page views of 145,063 Wikipedia web pages from Jul. 1st, 2015 to Dec. 31st, 2016.
To preprocess it for our experiments, we remove the web pages with empty page view record on any day (117,277 left), and compute the mean page views across all dates for each web page.
The goal is to release the page views (i.e., a 117,277-dimensional vector) while protecting the mean of the distribution (which reveals the business scales of the company § 1).
Baselines.
We compare our mechanisms discussed in § 4 with two popular mechanisms proposed in prior work (§ 1): differentially-private density estimation [22] (shortened to DP) and attribute-private
Gaussian mechanism [26] (shortened to AP).
For a dataset of samples X = { x 1 , ..., x n } , DP works by: (1) Dividing the space into m bins:
B 1 , ..., B m .
(2) Computing the histogram C
i = (cid:80) nj
=1 I ( x j ∈ B i ) .
Adding noise to the histograms D i = max { 0 , C i + Laplace (0 , ϵ ) } , where Laplace (cid:0) 0 , ϵ 2 (cid:1) means a random noise from
Laplace distribution with mean 0 and variance ϵ 2 .
(4) Normalizing the histogram p i =
D i (cid:80) mj
=1 D j .
We can then draw y i according to the histogram and release Y =
{ y 1 , ..., y n } with differential privacy guarantee.
AP works by releasing Y = (cid:8) x
i + N (cid:0) 0 , ϵ 2 (cid:1)(cid:9) n i =1 .
Metrics.
Our privacy and distortion metrics depend on the prior distribution of the original data θ ∼ ω Θ (though the mechanism does not).
In practice (and also in this experiment), the data holder only has one dataset.
Therefore, we cannot empirically evaluate the proposed privacy and distortion metrics, and resort to surrogate metrics to bound our true privacy and distortion.
Surrogate privacy metric.
For an original dataset X = { x 1 , ..., x n } and the released dataset
Y = { y 1 , ..., y n } , we define the surrogate privacy metric ˜Π ϵ as the error of an attacker who guesses the mean of the released dataset as the secret: ˜Π ϵ ≜ − (cid:12)(cid:12) 1 n (cid:80) ni
=1 x i − 1 n (cid:80) ni
=
y i (cid:12)(cid:12) .
Note that a minus sign is added here so that a smaller value indicates stronger privacy, as in privacy metric (1).
This simple attacker strategy is in fact a good proxy for evaluating the privacy Π ϵ due to the following facts.
(1) For our mechanism Alg.
1,when the prior distribution is uniform, this strategy is actually optimal (see § 4), so there is a direct mapping between ˜Π ϵ and Π ϵ .
(2) For AP, this strategy gives an unbiased estimator of the secret.
(3) For DP, this mechanism may not be an unbiased estimator of the secret, but it gives an upper bound on the attacker’s error.
Surrogate distortion metric.
We define our surrogate distortion metric as the Wasserstein-1 distance between the two datasets: ˜∆ ≜ d Wasserstein-1 ( p X ∥ p Y ) where p D denotes the empirical distribution of a dataset D .
This metric evaluates how much the mechanism distorts the dataset.
[Figure]
) of AP, DP and ours on Wikipedia Web Traffic Dataset.
Each point represents one instance of data release mechanism with one hyper-parameter.
In fact, we can deduce a theoretical lower bound for the surrogate privacy and distortion metrics using similar techniques as Corollary 2 (App. E).
Results.
We enumerate the hyper-parameters of each method (bin size and ϵ for DP, ϵ for AP, and s for ours).
For each method and each hyper-parameter, we compute their surrogate privacy and distortion metrics.
The results are shown in Fig.
3 (bottom left is best); each data point represents one realization of mechanism M g under a distinct hyperparameter setting.
AP directly adds Gaussian noise to each sample.
This process does not change the mean of the distribution on expectation.
Therefore, Figure 3 shows that AP has a bad privacydistortion tradeoff.
DP quantizes (bins) the samples before adding noise.
Quantization has a better property in terms of protecting the mean of the distribution, and therefore we see that DP has a better privacy-distortion tradeoff than AP.
Our mechanism (Eq.
(4)) always achieves the best privacy-distortion tradeoff; indeed, it can be shown to be optimal even for these surrogate metrics (proofs in App. E).
Discussions and Future Work
This paper gives an initial framework for defining, analyzing, and protecting distributional privacy.
Many open questions remain, including how to measure privacy empirically for arbitrary secrets and data distributions, how to analyze the privacy-distortion tradeoff (and design mechanisms) for more secret types, distribution classes (including parametric families without an explicit location parameter), multiple secrets, higher dimension of data, and other distance metrics for distortion.
While our framework is not intended for protecting sample level privacy (see § 1), it would be interesting to study whether our mechanisms (§ 4) are able to defend against sample-level attacks (e.g., membership inference attacks [10]) as a side benefit.
Acknowledgement
This research was sponsored in part by the National Science Foundation RINGS award 2148359 and by the Air Force Office of Scientific Research under award number FA9550-21-1-0090.
This work was also supported in part by faculty research awards from JP Morgan Chase, Intel, and the Sloan
Foundation, as well as gift grants from Cisco and Siemens AG.
This material is based upon work supported by the U.S. Army Research Office and the U.S. Army Futures Command under Contract
No. W911NF20D0002.
The content of the information does not necessarily reflect the position or the policy of the government and no official endorsement should be inferred.
Appendix A Related Work
In this section, we discuss why existing privacy frameworks and mechanisms are unsuitable for this distributional privacy concern.
Differential privacy (DP) [6] is one of the most popular privacy notions.
A randomized mechanism
M is ( ϵ, δ ) -differentially-private if for any neighboring datasets D 0 and D 1 (i.e., D 0 and D 1 differ one sample), and any set S ⊆ range ( M ) , we have
[Equation]
In our data sharing scenarios, we could apply DP framework by treating M as the data release mechanism that reads the original dataset and outputs the released dataset.
However, the privacy concerns of DP and our suggested framework are completely different: we aim to hide functions of a distribution , while DP aims to hide whether any given sample contributed to the shared data.
Indeed, we will show in § 5 that DP is not effective in hiding distributional secrets.
Maximal leakage [12] is a framework for quantifying the leakage of sensitive information.
We denote X as the random variable of the data that many contain sensitive information, and Y as the random variable of the information that is processed from X and is accessible by the attacker.
With the observation of Y , the attacker’s goal is to guess a secret function of X denoted by U , and the guess is denoted by ˆ U .
Based on this setup, the Markov chain U − X − Y − ˆ U holds.
Maximal leakage from X to Y is defined as
[Equation]
where the sup is taking over U (i.e., considering the worst-case secret) and ˆ U (i.e., considering the strongest attacker).
Intuitively, Eq.
(5) evaluates the ratio (in nats) of the probabilities of guessing the secret correctly with and without observing Y .
To apply maximal leakage in data sharing scenario, we may regard X as the original dataset, Y as the released dataset, and U as the secret (e.g., the fraction of a specific server type).
However, this formulation is still unsuitable for the following reasons.
(1) Maximal leakage only considers discrete
U and ˆ U under finite alphabet.
Note that it is a critical assumption for making sure that P (cid:16) U
=
ˆ U (cid:17) in the definition (Eq.
(5)) is nontrivial.
However, in our problem, secrets are usually in continuous domains (e.g., the secrets in the motivating examples in § 1).
(2) Maximal leakage assumes that the secret to protect U is unknown a priori and therefore considers the worst-case leakage among all possible secrets.
However, in our problem, data holders know what secret they want to protect.
Attribute privacy [26] considers a similar privacy concerns as us: it tries to protect a function of a sensitive column in the dataset (named dataset attribute privacy ) or a sensitive parameter of the underlying distribution from which the data is sampled (named distribution attribute privacy ).
Attribute privacy applies Pufferfish privacy framework [14] to describe these two privacy concerns.
On the high level, an algorithm is said to be of higher dataset/distribution attribute privacy if for any two different ranges of secret (e.g., fraction of the server type is in [0 . 1 , 0 . 2) or [0 . 2 , 0 . 3) ), the distributions of the algorithm output do not differ too much.
For the more accurate definitions, please refer to [26].
Although the privacy concerns are highly related, attribute privacy focuses on algorithms that output a statistical query of the dataset (e.g., the number of servers) instead of the entire dataset (e.g., cluster trace dataset).
We could still apply the framework to analyze such data sharing algorithms, but due to the high dimensionality of the dataset, attribute privacy requires to add too much noise which makes the generated dataset of high distortion.
Indeed, in § 5 we will see that attribute privacy mechanisms give poor privacy-distortion trade-offs in data sharing applications.
Privacy funnel [16] is another popular privacy framework.
We denote X as the random variable of the data that many contain sensitive information U , and Y as the random variable of the information that is processed from X and is accessible by the attacker.
The privacy funnel framework use the mutual information
I ( U ; Y ) to evaluate the privacy leakage, and use the mutual information I ( X ; Y ) , to evaluate how well Y preserves the utility of data.
To find a good privacy-preserving data processing strategy P Y | X , privacy funnel solves the optimization problem
[Equation]
where R is a desired threshold on the utility of Y .
To apply it in data sharing problems, we could regard X as the original data, Y as the released data, and U as the secret data holder wants to protect (e.g., the fraction of a specific server type).
However, mutual information is not a good metrics for either privacy or utility.
On the privacy front, prior work has shown that we could make I ( S ; Y ) smaller while allowing the attacker to guess S correctly from
Y with higher probability (see Example 1 in [12]).
On the utility front, higher mutual information
I ( X ; Y ) does not mean that the released data Y is a useful representation of X .
For example, for could have Y as an arbitrary one-to-one transformation of X .
In that case, I ( X ; Y ) obtains the maximum but the structure of data could be completely destroyed.
In addition, privacy funnel [16] only considers X and Y in discrete domains, which is not the case for our setting.
Sub-sampling is used in prior data release practice for reducing the leakage of sensitive information [18].
It works by sub-sampling the original datasets before doing follow-up processing or release.
The intuition is that by reducing the number of involved samples, less information can be leaked.
However, sub-sampling does not change the statistical properties of the distribution.
Distributional information can still be inferred from the released data (possibly with higher estimation variance, though).
Anonymization , such as removing certain attributes (e.g., name of the patients in medical data, name of jobs in cluster dataset) [18], is widely used in the release process of public datasets (e.g., [23]).
However, this process does not change the distribution of other attributes, which may contain the private information that data holder wants to hide [15].
Summary.
Based on the above observation, it is important to establish a new privacy framework for distributional privacy in data sharing scenarios, which we discuss in the next section.
B Proof of Thm.
Proof.
For any θ ′ , we have
[Equation]
where Eq.
(6) comes from triangle inequality.
Let
[Equation]
From the above result
[Equation]
we know that R θ ′ − L θ ′ ≤
Tγ .
We can define a sequence of attackers such that ˆ g i ( θ ′ )
=
L θ ′
+ ( i + 0 . 5) · 2 ϵ for i ∈ (cid:110) 0 , 1 , . . .
, ⌈ T 2 γϵ ⌉ − 1 (cid:111) (Fig. 4).
We have
[Equation]
and therefore,
[Equation]
+2% ! ! !
+4% ! ! !
+ ' 2(% 2% +' ) +' " +' * +,- ."
Possible range of '(,)
Range of '(,) that +' ) succeeds Range of '(,) that +' " succeeds
Range of '(,) that +' !"
#$ ." succeeds which implies that
[Equation]
Therefore, we have
[Equation]
C Proof of Corollary
Proof.
For any X θ 1 , X θ 2 , we have
[Equation]
where Eq.
(8) comes from Jensen inequality.
Therefore, we have γ = inf θ
[Equation]
D Proof of Thm.
Proof.
For any released parameter θ ′ =
( u ′ , v ′ ) , there exists i ∈ { 0 , ..., N − 1 } such that u ′ =
u
+ ( i + 0 . 5) · s .
We have
[Equation]
Therefore
[Equation]
we have
[Equation]
θ ′ (cid:19)(cid:33) ≤ 2 ϵ s .
Let ˆ g ∗ be the attack strategy that ˆ g ∗ ( u ′ , v ′ )
=
u ′ .
When ˆ g = ˆ g ∗ , the equality achieves, and thus ˆ g ∗ ∈ arg sup ˆ g P
(ˆ g ( θ ′ ) ∈
[ g ( θ ) − ϵ, g ( θ ) + ϵ ]) .
For the distortion, we can easily get that ∆ =
s 2 .
From Corollary 1, we know that Π ϵ ≥ ⌈ s 2 ϵ ⌉ − 1 .
Therefore, when 2 ϵ divides s , the data release mechanism in Alg. 1 is optimal.
E Proofs for the Surrogate Metrics
For any p Y , we have ˜∆ = d Wasserstein-1 ( p X ∥ p Y ) ≥ (cid:12)(cid:12) 1 n (cid:80) ni
=1 x i − 1 n (cid:80) ni
=
y i (cid:12)(cid:12) =
− ˜Π ϵ .
For p Y is released from our mechanism Eq.
(4), we have ˜ ∆ =
d Wasserstein-1 ( p X ∥ p Y ) =
(cid:12)(cid:12) 1 n (cid:80) ni
=1 x i − 1 n (cid:80) ni
=
y i (cid:12)(cid:12) =
− ˜Π ϵ .
F Analysis of the Alternative Formulation
In this section, we present the alternative formulation of minimizing distortion ∆ subject to a constraint on privacy metric Π ϵ : min M g ∆ subject to Π ϵ ≤ T (9)
Theorem 3 (Lower bound of privacy-distortion tradeoff) .
Let D ( X θ 1 , X θ 2 ) ≜ 1 2 d (cid:0) ω
X θ 1 ∥ ω
X θ 2 (cid:1) , where d ( ·∥· ) denotes Wasserstein-
α distance.
Further, let R ( X θ 1 , X θ 2 ) ≜ | g ( θ 1 ) − g ( θ 2 ) | .
Let γ ≜ inf θ 1 ,θ 2 ∈ Supp ( ω Θ ) D
( X θ 1 ,X θ 2 ) R
[Table]
X
[Table]
X
[Table]
For any T ∈ [
[Table]
when Π ϵ ≤ T , we have ∆
[Table]
The proof is in App.
G.
[Figure] [Figure]
[Figure] [Figure]
[Figure]
Corollary 2 (Privacy lower bound, secret = mean of a continuous distribution) .
Consider the secret function g ( θ )
= (cid:82) x xf X
θ ( x ) dx .
For any T ∈ [0 , 1) , when Π ϵ ≤ T , we have ∆ > (cid:0) ⌈ 1 T ⌉ − 1 (cid:1) · ϵ .
The proof is similar to App.
C except that the last step applies Thm.
3 instead of Thm.
Theorem 4 (Optimality of Alg. 1 ) .
The data release mechanism in Alg.
1 asymptotically achieves the lower bound in Thm.
3 when ϵ → 0 .
The proof is in App.
H. Let ˆ g ∗ be the attack strategy that ˆ g ∗ ( u ′ , v ′ )
=
u ′ , i.e., the attack guesses the mean of private distribution as the release mean.
We also show in the proof that under the data release mechanism Alg. 1, ˆ g ∗ is the optimal attacker strategy (i.e., ˆ g ∗ ∈ arg sup ˆ g P
(ˆ g ( θ ′ ) ∈
[ g ( θ ) − ϵ, g ( θ ) + ϵ ]) ).
G Proof of Thm.
Proof.
[Equation]
Therefore, there exists θ ′ s.t.
sup ˆ g P
[Equation]
g
[Equation]
∈ [ g
[Equation]
) −
[Equation] [Equation]
ϵ ]
[Equation]
θ ′
[Equation]
≤ T .
Let
[Equation]
We can define a sequence of attackers such that ˆ g i ( θ ′ )
=
L θ ′
+( i + 0 . 5) · 2 ϵ for i ∈ { 0 , 1 , . . .
, N − 1 } and L θ ′
+
Nϵ ≥ R θ ′ >
L θ ′ + 2( N − 1)
ϵ (Fig. 5).
From the above, we have
[Equation]
Therefore, we have N ≥ ⌈ 1 T ⌉ , and R θ ′ − L θ ′ > (cid:0) ⌈ 1 T ⌉ − 1 (cid:1) · 2 ϵ .
Then we have
[Equation]
where Eq.
(11) utilizes R θ ′ − L θ ′ > (cid:0) ⌈ 1 T ⌉ − 1 (cid:1) · 2 ϵ and the definition of γ , and Eq.
(10) comes from triangle inequality.
H Proof of Thm.
Proof.
For any released parameter θ ′ =
( u ′ , v ′ ) , there exists i ∈ { 0 , ..., N − 1 } such that u ′ =
u
+ ( i + 0 . 5) · s .
We have
[Equation]
s
[Equation]
Therefore, we have
[Equation]
Let ˆ g ∗ be the attack strategy that ˆ g ∗ ( u ′ , v ′ )
=
u ′ .
When ˆ g = ˆ g ∗ , the equality achieves, and thus ˆ g ∗ ∈ arg sup ˆ g P
(ˆ g ( θ ′ ) ∈
[ g ( θ ) − ϵ, g ( θ ) + ϵ ]) .
For the distortion, we can easily get that ∆ =
s 2 .
Corollary 2 says that ∆ > (cid:0) ⌈ s 2 ϵ ⌉ − 1 (cid:1) · ϵ .
The gap between these two goes to zero when ϵ → 0 .