Bridging Central and Local Differential Privacy in Data Acquisition Mechanisms
Abstract
We study the design of optimal Bayesian data acquisition mechanisms for a platform interested in estimating the mean of a distribution by collecting data from privacy-conscious users.
In our setting, users have heterogeneous sensitivities for two types of privacy losses corresponding to local and central differential privacy measures.
The local privacy loss is due to the leakage of a user’s information when she shares her data with the platform, and the central privacy loss is due to the released estimate by the platform to the public.
The users share their data in exchange for a payment (e.g., through monetary transfers or services) that compensates for their privacy losses.
The platform knows the distribution of privacy sensitivities but not their realizations, and must design a mechanism to solicit their preferences and then deliver both local and central privacy guarantees while minimizing the estimation error plus the expected payment to users.
We first establish minimax lower bounds for the estimation error, given a vector of privacy guarantees, and show that a linear estimator is (near) optimal.
We then turn to our main goal: designing an optimal data acquisition mechanism.
We establish that the design of such mechanisms in a Bayesian setting (where the platform knows the distribution of users’ sensitivities and not their realizations) can be cast as a nonconvex optimization problem.
Additionally, for the class of linear estimators, we prove that finding the optimal mechanism admits a Polynomial Time Approximation
Scheme.
Introduction
Users’ personal data are currently being utilized for personalized advertising, medical trials, targeted advertising, and recommendation systems, among others.
The transaction of individual data is set to grow exponentially in the coming years, with more widespread applications of artificial intelligence (AI) and machine learning techniques.
Even though it is widely accepted that users need to own their data (see, e.g., Posner and Weyl [2019], Kushmaro [2021], and WILL.I.AM [2019]), the impact of different market architectures on the design and operation of data markets are not clear: some users prefer to protect their own raw data while others expect companies to protect their data proactively (see, e.g., GDMA [2018]).
In this paper, we consider the design of data acquisition mechanisms when users have heterogeneous privacy concerns and ask the following question:
What is the optimal data acquisition mechanism when users have heterogeneous privacy concerns regarding access to their raw data and the outcome of the platform’s processing?
We use differential privacy to measure the two types of privacy losses.
Informally, an estimator is called differentially private if its distribution over outputs is insensitive to the changes in a user’s data.
In particular, we consider a platform whose goal is to estimate an underlying parameter of interest by collecting data from a set of users N = { 1 , . . . , n } who own a noisy version of the underlying parameter.
For instance, consider a medical trial in which a hospital wants to collect users’ data to estimate the efficacy of a drug.
Each user has two types of privacy concerns: (i) local privacy concern that captures how much information their shared data reveal about their raw data, and (ii) central privacy concern that captures how much information the platform’s output reveal about their raw data.
We adopt local and central Rényi differential privacy to measure these two types of privacy losses.
The reason for choosing Rényi differential privacy is twofold.
First, our framework can cover a wide range of information measures by varying the Rényi divergence parameter.
Second, it can be achieved by a Gaussian mechanism, which simplifies our analysis while capturing the main tradeoffs in the design of two-part data acquisition mechanisms.
Before formulating the platform’s data acquisition problem, we derive optimal estimators for a given vector of heterogeneous local privacy loss levels.
In particular, we establish a minimax lower bound for the estimation error and prove that first privatizing users’ data by adding a properly designed
Gaussian noise to them and then using a properly designed weighted average of these privatized data points achieves this lower bound.
This result motivates us to consider the design of the optimal data acquisition mechanism for the class of linear estimators.
We then turn to our mechanism design problem.
Each user has a heterogeneous preference regarding the importance of the above two privacy concerns.
For instance, if a user fully trusts the platform, then the first type of concern lessens, and the main concern would be about the information revealed from the platform’s estimate.
On the other hand, if a user does not trust the platform at all, the first type of concern would be more than the second one.
We model such a setting by assuming each user i has a privacy sensitivity c i ∈ [0 , 1] that determines the relative weight she puts on the local privacy concern ( 1 − c
i is the weight she puts on the central privacy concern).
The utility of user i is the payment she receives from the platform (in exchange for sharing her data), minus c
i times her local privacy loss, and again, minus 1 − c i times her central privacy loss.
The platform does not know the value of c i and (knowing its distribution) must design a (Bayesian) data acquisition mechanism to elicit the true privacy sensitivities (that guide the optimal choice of local and central privacy losses delivered to each user) and optimize its objective.
In particular, the platform designs a two-part data acquisition mechanism that comprises a payment scheme, a local privacy guarantee, and a central privacy guarantee as a function of the reported privacy sensitivity of users.
The platform’s goal is to minimize the sum of the mean estimation error and the expected total payment to users while satisfying the incentive compatibility and individual rationality constraints.
Incentive compatibility ensures that users have no incentive to misreport their privacy sensitivity, and individual rationality ensures that the payment to users (and the delivered privacy guarantees) are such that users are willing to share their data with the platform.
The platform’s problem is a functional optimization over three functions of the reported privacy sensitivities: payments, local privacy guarantees, and central privacy guarantees.
We first find the payment function in terms of the local and central privacy guarantees by using the incentive compatibility and individual rationality constraints.
This reduces the space of the platform’s decision variables.
We then focus on the Gaussian mechanisms and linear estimators (motivated by our minimax optimality result) and show that the platform’s problem can be cast as an optimization problem that minimizes a non-convex objective (which depends on the virtual cost of users) for any reported vector of privacy sensitivities.
This reformulation significantly reduces the space of decision variables that the platform needs to optimize.
However, it still involves solving a non-convex optimization problem.
We further use the structural properties of this non-convex optimization and use duality theory to develop a polynomial time algorithm to approximate the platform’s problem.
More precisely, we prove that the design of the optimal two-part data acquisition mechanism admits a Polynomial Time Approximation Scheme (PTAS).
The contribution of our work is threefold.
First, we develop a minimax lower bound when users have heterogeneous local privacy losses and establish that a linear estimator (approximately) achieves this bound.
Second, we develop a modeling framework for data acquisition mechanisms when users have heterogeneous concerns for both local and central privacy losses.
Third, for any estimator and mechanism to deliver privacy guarantees, we characterize the design of the optimal two-part data acquisition mechanism as the solution to a point-wise optimization problem.
Additionally, for the class of Gaussian mechanisms to deliver privacy guarantees and linear estimators, we develop an algorithm to approximately find the optimal data acquisition mechanism (despite the fact that the corresponding optimization is non-convex).
Related literature: Our paper relates to the literature on optimal data acquisition from privacy concerned users.
There is a large body of work that use differential privacy to measure the privacy loss of users Ghosh and Roth [2011], Nissim et al.
[2012], Nissim et al.
One of the earliest papers in the literature is Ghosh and Roth [2011], which study the design of mechanisms for collecting users’ data when users incur some privacy cost from sharing their data.
More specifically, Ghosh and Roth [2011] consider binary data (bit) with the platform’s goal being to estimate the sum of user’s data by using a differentially private and dominant strategy truthful mechanism.
They study both the case when the user data and privacy parameter are independent (similar to our paper) and when they are correlated.
In the independent case, they propose a mechanism that delivers a single privacy level to all users (as opposed to our setting that delivers heterogeneous privacy levels).
For the correlated case, they prove an impossibility result for the existence of a truthful and individually rational mechanism.
Several works build on Ghosh and Roth [2011], extending it to take it or leave it offers Ligett and Roth [2012], strengthening the impossibility results Nissim et al.
[2014], and studying the open question posed by Ghosh and Roth [2011] on whether a model with distributional assumption on users’ costs and Bayesian mechanism design approach could be used to develop the optimal mechanism for collecting data with privacy guarantees (see, e.g., Liao et al.
[2021] and Fallah et al.
In particular, Roth and Schoenebeck [2012], Chen et al.
[2018], and Chen and Zheng [2019] tackle this problem by developing a randomized mechanism in which user’s data is randomly included in the final estimator where the inclusion probability depends on the reported privacy costs of the users (as opposed to our setting in which the payments and privacy guarantees depend on the reported privacy sensitivity of all users).
These papers do not use differential privacy to model privacy costs and instead use a menu of probability-price pairs to tune the privacy loss and the payment to each user (see also Pai and Roth [2013] for a survey).
Similar to the above paper, we consider a setting in which the platform can verify the data of users.
For instance, in the context of medical trials, this means that the users decide whether to participate in the medical trial and cannot change the samples they share.
A different stream of the literature explores settings in which users can misreport their information
Perote and Perote-Pena [2003], Dekel et al.
[2010], Meir et al.
[2012], Ghosh et al.
[2014], Cai et al. [2015], Liu and Chen [2016, 2017].
Our paper differs from these works in three main ways.
First, we assume prior information on user privacy sensitivities and focus on characterizing the optimal Bayesian incentive compatible mechanism.
Second, we model a setting in which users have both local and central privacy concerns and explore the different privacy guarantees of these two types delivered by an optimal mechanism.
Third, we assume that user data are drawn from the same underlying distribution that allows the platform to put differing weights on the data of users depending on their privacy sensitivity, leading to different privacy levels for participating users.
Finally, our paper relates to the literature on differential privacy.
Pioneered by the seminal work of
Dwork et al.
[2006a,b], differential privacy has emerged as a prevalent framework for characterizing the privacy leakage of data oriented algorithms.
More specifically, our paper is related to the private mean estimation considered by Duchi et al.
[2013], Barber and Duchi [2014], Karwa and Vadhan [2017], paper relates to the stream of differential privacy literature that studies
Rényi differential privacy (RDP) introduced by Bun and Steinke [2016] and Mironov [2017].
Problem Formulation
We consider a platform interested in estimating a parameter θ ∈ R by collecting data of n users, indexed by N = { 1 , · · · , n } .
For any i ∈ N , we denote user
i ’s data by X
i ∈
X
and we assume X i is given by X i
= θ
+ Z i where Z 1 , · · · , Z n are independent and identically distributed zero-mean random variables with variance VAR .
To simplify the exposition, we further assume | Z i | ≤ 1 / 2 for any i ∈ N .
Throughout, we use lower case letters to denote the realization of random variables.
The platform’s goal is to minimize the estimate’s error by collecting data from privacy-concerned users.
Therefore, the platform needs to incentivize them to share their data.
2.1 Local and central privacy losses
Before formalizing the utilities/objectives of the platform and the users, we define the notions of privacy losses that we adopt in this paper.
In particular, we consider two different types of privacy losses that users suffer from.
The first one is the privacy loss of a user when she shares her data (only) with the platform, and the second one is the privacy loss through the released estimate (to the public) by the platform.
Depending on how different users trust the platform, they might care differently about these two privacy losses.
For instance, if a user fully trusts the platform, then her main privacy concern would be the second one, while a user who does not trust the platform at all would be more concerned with the first one as the public only observes the aggregated estimate, as opposed to the platform which observes each user’s (shared) data separately.
We use the differential privacy framework to quantify these privacy losses.
Since differential privacy was introduced by Dwork et al., several variants of it have been also proposed.
In particular, a popular one in the machine learning literature is Rényi differential privacy (RDP), introduced by Mironov [2017], which we also adopt in this paper.
Let us first recall the definition of Rényi divergence.
Definition 1.
Let P and Q be two distributions over R with densities p and q .
For any α ∈ (1 , ∞ ] , the Rényi α -divergence between P and Q is denoted by D α ( P || Q ) and is given by
[Equation]
For two random variables X and Y , D α ( X || Y ) denotes the α -divergence between their distributions.
We next define two notions of differential privacy, known as central and local, to capture the two aforementioned types of privacy losses.
Local differential privacy corresponds to the privacy loss of a user when she shares her data with the platform through a randomized mapping, known as a channel .
Definition 2.
Let ε ≥ 0 and α ∈ (1 , ∞ ] .
A randomized channel C : X → R is locally ( ε, α ) -Rényi (differentially) private if for any x, x ′ ∈ X ,
[Equation]
Central differential privacy corresponds to the other privacy loss mentioned above.
It bounds the change in the distribution of the platform’s output, i.e., the released estimate, by changing one user’s data.
We next provide the formal definition.
Definition 3.
Let ε
=
( ε i ) ni
=1 ∈ R n + and α ∈ (1 , ∞ ] .
A randomized algorithm A : X n → R is ( ε , α ) -Rényi (differentially) private if for any two datasets x 1: n , x ′ 1: n ∈ X n that only differ in i -th coordinate (data of user i ),
[Equation]
The customary approach to guarantee RDP is Gaussian mechanism in which a properly tuned zeromean Gaussian noise is added to fulfill the required condition.
The following lemma, adapted from
Mironov [2017], allows us to characterize the Gaussian noise’s variance for a privacy loss level.
Lemma 1.
For a function f : X n → R , we define its sensitivity with respect to i -th coordinate as
L i ( f ) :
= sup {| f ( x 1: n ) − f ( x ′ 1: n ) | : for all x 1: n and x ′ 1: n differing only at i -th coordinate } .
For any α ∈ (1 , ∞ ] , A ( x 1: n ) = f
( x 1: n ) + W with W ∼ N (0 , σ 2 ) is (cid:16) ( αL i ( f ) 2 2 σ 2 ) ni
=1 , α (cid:17) -RDP.
For a given vector of local privacy losses ( ε ( l ) 1 , . . .
, ε ( l ) n ) , a natural way to privately estimate the mean is by using a linear estimator with Gaussian mechanism which is given by ˆ θ ( x 1 , . . .
, x n ) := n (cid:88) i =
w i ˆ x i where n (cid:88)
i =
w i = 1 and ˆ x i
= x
i + N (0 , α 2 ε ( l ) i ) for all i ∈ N .
Figure 1 depicts this estimator.
In particular, using Lemma 1, the local privacy delivered to user i ∈ N is ε ( l ) i and the central privacy delivered to user i ∈ N is ε ( c ) i
[Equation]
In the next section, we focus on linear estimators in designing the optimal private data acquisition mechanism.
We motivate such specification by showing that, for a given vector of local privacy losses ( ε ( l ) i ) ni =1 , a linear estimator is optimal with respect to mean square error.
To formalize this statement, we first need to define the minimax estimation error as the notion of optimality.
Let P be a class of distributions over X .
For any P ∈ P , we denote its mean by θ ( P ) .
A ( ε ( l ) i ) n i =1 -locally RDP estimator can be cast as ˆ θ (( C i ( x i ))
ni =1 ) , where C i ( . ) is the randomized channel corresponding to user i .
Let
Q (( ε ( l ) i ) ni =1 ) be the class of such ( ε ( l ) i ) ni =1 -locally RDP estimators.
The minimax estimation error is
L ( P , Q , ( ε ( l ) i ) ni =1 ) :=
[Equation]
where the expectation is taken over both randomness of data and estimator (including private channels).
In other words, the optimal estimator is the one that has the lowest worst case error among all estimators that satisfy the privacy requirements.
We next state the optimality result.
Theorem 1.
Assume α ≥ 2 and ε ( l ) i ≤ 1 for all i .
Let P 1 be the family of distributions over [ − 12 , 12 ] and C 1 , · · · , C n be independent channels.
Then, there exists a universal constant c such that
[Equation]
Furthermore, there exists a linear estimator with Gaussian mechanism such that
[Equation]
We prove the lower bound by using the Le Cam’s method Yu [1997] that reduces the problem of finding lower bounds to a hypothesis testing problem and requires bounding the total variation distance between the estimates when the input data points change.
We then find it more convenient to bound the total variation in terms of Hellinger distance and use a series of inequalities to bound it.
2.2 Data acquisition mechanism with two-part privacy guarantees
We next describe the utility functions of the users and the platform and then formulate the platform’s optimal data acquisition mechanism.
As we described earlier, each user suffers from two privacy losses when sharing her data.
The first one is a central privacy loss because of the leakage of her information through the platform’s output.
The second one is a local privacy loss because of the leakage of her information through the raw data that she shares with the platform.
Each user has a heterogeneous privacy sensitivity for these two types of privacy losses.
To model such heterogeneity, for each i ∈ N , we let c i ∈
[0 , 1] be her relative local privacy sensitivity , representing the relative weight that user i assigns to the (per unit cost of) local privacy loss.
We also let 1 − c
i be her relative central privacy sensitivity , representing the relative weight that user i assigns to the (per unit cost of) central privacy loss.
Therefore, c i ≈ 1 implies that user i suffers a higher loss of privacy by sharing her raw data with the platform (local privacy loss) compared to her loss from the platform’s output (central privacy loss).
Differently, c i ≈ 0 implies that the user suffers a smaller loss of privacy by sharing her raw data with the platform compared to her loss from the platform’s output.
In what follows, we use the term privacy sensitivity instead of relative local privacy sensitivity.
For each i ∈ N , the privacy sensitivity c i is independently drawn from a publicly known distribution whose support is [0 , 1] with cumulative distribution and probability density functions F i ( · ) and f i ( · ) .
We also let c = ( c 1 , . . .
, c n ) be the vector of privacy sensitivities.
The privacy sensitivity of each user is her private information, i.e., the platform does not know it.
This is because individuals have different views regarding how trustworthy the platform is in protecting their raw data.
The platform’s objective is to design a mechanism to collect users’ data by paying them to compensate for their privacy losses without knowing the privacy sensitivity of users.
To introduce the platform’s objective formally, we adopt the formalism of Bayesian mechanism design pioneered by Myerson [1981].
More specifically, the platform designs and announces a payment function, a local privacy loss function, and a central privacy loss function that are mappings from the reported privacy sensitivities of users.
The users then report their privacy sensitivities (which may or may not be truthful).
Based on the payment function, the platform compensates the users (the compensation could be monetary or some free or discounted service provided to the user).
Based on the local and central privacy functions, the platform designs randomized channels and randomized estimation algorithms that deliver the guaranteed local and central privacy losses while minimizing the sum of the mean squared error and the total expected payments.
Given this interaction, we next formally introduce a data acquisition mechanism with two-part data privacy guarantees.
Definition 4 (two-part private data acquisition mechanism) .
We call the tuple (ˆ θ, ε ( l ) , ε ( c ) , t ) a two-part private data acquisition mechanism where
For all i ∈ N , ε ( l ) i : R n + → R + is a function that maps the vector of privacy sensitivities c to a local privacy loss for user i , ε ( l ) i ( c ) , with ε ( l ) ( · ) =
( ε ( l ) i ( · ))
ni =1 . 2.
For all i ∈ N , ε ( c ) i : R n + → R + is a function that maps the vector of privacy sensitivities c to a central privacy loss for user i , ε ( c ) i ( c ) , with ε ( c ) ( . )
= ( ε ( c ) i ( · ))
ni =1 .
ˆ θ : X n × R n + × R n + → R is a ( ε ( c ) ( c ) , α ) -Rényi differentially private estimator that maps acquired locally ( ε ( l ) i ( c ) , α ) -Rényi differentially private data of user
i for i ∈ N to an estimate ˆ θ ( x , ε ( l ) ( c ) , ε ( c ) ( c )) .
For all i ∈ N , t i : R n + → R + is a function that maps the vector of privacy sensitivities c to a payment for user i , t i ( c ) , with t ( . )
=
( t i ( · )) ni =1 .
Notice that we have not specified the estimator and the mechanisms that delivers (local and central)
Rényi differential privacy.
In the rest of this subsection, we introduce the utilities and the platform’s problem for a general estimators and mechanisms to deliver differential privacy.
Later, we focus on linear estimator and Gaussian mechanisms and explicitly solve the platform’s problem.
Each user that participates in a two-part private data acquisition mechanism suffers from both the local and central privacy losses and need to be compensated by the platform.
In particular, the utility of user i from participation when her privacy sensitivity is c i
and she reports c
i is given by u i
[Equation]
i ,
[Equation]
[Equation]
] , where the term t i ( c − i , c ′ i )) is the payment from the platform, the term c i ε ( l ) i ( c − i , c ′ i ) is the relative local privacy sensitivity of the user multiplied by her local privacy loss, and the term (1 − c i ) ε ( c ) i ( c − i , c ′ i ) is her relative central privacy sensitivity multiplied by her central privacy loss.
A user i ∈ N that does not participate in the mechanism neither compromises her privacy nor gets a compensation.
Therefore, the utility of a user who does not participate in the mechanism becomes 0 .
The goal of the platform is to minimize the sum of the mean squared error and the overall payment to users.
We let γ ∈ R + represents the relative weight of the mean estimation error and the payments in the platform’s objective.
Therefore, the platform’s objective is
[Equation]
where the first term is the mean square error of estimator ˆ θ given reported vector of privacy sensitivity and resulting local and central privacy losses ε ( l ) and ε ( c ) , i.e.,
[Equation]
Also, each summand of the second term is the compensation that the platform gives to a user to incentivize her to participate and report her privacy sensitivity truthfully.
In Appendix we prove that, similar to the classical mechanism design setting, revelation principle holds.
This means that there is no loss of generality in focusing on the class of direct incentive compatible mechanisms, meaning the platform’s optimization problem can be written as
[Equation]
where the constraints in (4) represent the incentive compatibility .
These constraints guarantee that that each user i has no incentive to misrepresent her privacy sensitivity when others report truthfully (reporting truthfully is an equilibrium of the game among the users).
Also, the constraints in (5) represent individual rationality , which ensures that each user receives a non-negative utility from participating in the platform’s mechanism and sharing her data.
From the mechanism design problem to an optimization problem
For a given estimator ˆ θ , the platform’s decision comprises the local and central privacy loss functions ε ( l ) ( · ) and ε ( c ) ( · ) together with the payment functions t ( · ) .
We next show that this problem can be equivalently formulated as an optimization problem over the vector of local privacy losses and central privacy losses (as opposed to functions).
In the rest of the paper, we impose the following assumption which is well-known in the mechanism design literature and simplifies the analysis.
Assumption 1.
For any user i ∈ N , the virtual cost defined as ψ i ( c ) =
c
+ F i ( c ) f i ( c ) is increasing in c , where f i ( · ) and F i ( · ) are probability density and cumulative distribution functions of c i , respectively.
The above assumption holds for a wide class of distributions such as the ones with log-concave density functions (e.g., uniform).
Theorem 2.
Suppose Assumption 1 holds.
For a given estimator ˆ θ : ˆ X n × R n + × R n + → R , in the optimal two-part data acquisition mechanism, for a given vector of reported privacy sensitivities c , the local and central privacy losses are the solution of
[Equation]
Proof Sketch of Theorem 2: We introduce the following interim functions t
[Equation]
We first establish a payment identity that determines the optimal payment in terms of the optimal local and central privacy losses.
In particular, by evaluating the first order condition corresponding to the incentive compatibility constraint (4), we establish that this constraint holds if and only if
[Equation]
and ε ( l ) i
( z ) − ε ( c ) i
( z ) is weakly decreasing in z .
We then plug in this payment identity back to the platform’s objective, use the individual rationality constraint, and rewrite the platform’s expected utility in terms of the privacy loss functions and the virtual cost of users.
This is still a functional optimization problem in terms of ε ( l ) ( · ) and ε ( c ) ( · ) .
However, we establish that, under Assumption 1, we can solve this functional optimization point-wise (i.e., for any given c ).
■
Theorem 2 highlights the tradeoff in the platform’s problem: by decreasing the local privacy loss, the second term of the objective decreases (this term corresponds to the payment to users) while the first term (i.e., the mean squared error) increases.
The role of the central privacy loss is more nuanced, and there are two cases.
If the coefficient 1 − ψ i ( c i ) is non-negative, by decreasing the central privacy loss, the third term of the objective decreases while the first term increases.
If the coefficient 1 − ψ i ( c i ) is negative, increasing the central privacy loss decreases both the third term and the first term.
However, we cannot increase the central privacy loss level without limits because the central privacy loss level is always below the local privacy loss level.
Therefore, the platform’s optimal mechanism should find the “right” balance between these terms.
Optimal mechanism with two-part privacy guarantees for linear estimators
For the rest of the paper, we focus on linear estimators with Gaussian mechanism described in Section 2.1.
The following is a direct corollary of Theorem 2.
Corollary 1.
Suppose Assumption 1 holds.
For any reported vector of privacy sensitivities c , the optimal local privacy loss levels are ε ( l ) i ( c ) =
y ∗ i and the optimal central privacy loss levels are ε ( c ) i =
w ∗ i 2 / n (cid:88)
j = w ∗ jy ∗ j where ( w ∗ 1 , . . .
, w ∗ n ) and ( y ∗ 1 , . . . , y ∗ n ) are the optimal solution of min w , y VAR γ n (cid:88)
i =
w 2 i + γαn (cid:88) i =
w 2 i y i + n (cid:88) i = (1 − ψ i ( c i ))
w 2 i (cid:80) nj =1 w 2 j y j + n (cid:88) i =
ψ i ( c i )
y i (7) s.t
w i , y i ≥ 0 , for all
i ∈ N and
n (cid:88) i =
w i = 1 .
Let us highlight the difference between our characterization and that of classic mechanism design (e.g.,
Myerson [1981]).
In classic mechanism design, the designer’s problem becomes linear optimization.
However, in our setting, the designer’s problem is a non-linear and non-convex optimization.
This makes the problem of finding the optimal two-part data acquisition mechanism challenging.
Before addressing this computational challenge, let us revisit the form of the Gaussian mechanism that we have adopted: the platform adds Gaussian noise locally and then outputs a convex combination of the privatized users’ data without adding any noise centrally.
More specifically, one may guess that the platform may benefit by having a central noise added to the final output in addition to the local noises.
In the following subsection, we establish that there is another Gaussian mechanism for any
Gaussian mechanism that only adds local noises and achieves a weakly lower cost.
4.1 Optimality of having only local noises in the Gaussian mechanism
The platform has the opportunity of adding Gaussian noise to both the raw data of each user and the final estimator and ex-ante one may guess that it is optimal to use both of these instruments.
However, as we establish next, interestingly, in the optimal two-part data acquisition mechanism, it is always optimal to only add noises locally.
For a given vector of local privacy losses ( ε ( l ) 1 , . . .
, ε ( l ) n ) and central privacy losses ( ε ( c ) 1 , . .
, ε ( c ) n ) , a
Gaussian mechanism with both local and central noises is of the form ˆ θ ( x 1 , . . . , x n ) :
[Equation]
where
[Equation]
[-w i = 
-]
Proposition 1.
In the optimal two-part data acquisition mechanism that adopts a Gaussian mechanism with both local and central noises, we have ε = ∞ .
Input: The vector of privacy sensitivities ( c 1 , . . . , c n ) for S ∈ Grid (cid:0) [ S, ¯ S ] , δ (cid:1) do Let ν i = γ VAR + (1 − ψ i ( c i )) /S , ζ i
=
ν i (cid:80) j ν j , ξ i = ζ i   n (cid:88) j = ν j ( (cid:113) ψ j ( c j ) − (cid:112) ψ
i ( c i ))   . Let p = (cid:18) (cid:80) ni =1 ζ i √ ψ
i ( c i ) S − (cid:80) ni =1 √ ψ i ( c i ) ξ i (cid:19). Let w i ( S )
=
ν i
+ ν i (cid:80)
j ν j (cid:112) ψ j ( c j ) p (cid:80) j ν j −
ν
i (cid:112) ψ i ( c i ) p, y i ( S ) =
w i ( S ) (cid:114) p ψ i ( c i ) , and OBJ( S ) be the objective of Problem (7) evaluated for this solution.
end Output: { y i ( S ∗ ) , w i ( S ∗ ) } n i =1 , where ( S ∗ ) =
arg min ( S ) OBJ( S ) .
Proposition 1 has an important implication in terms of the design of data market architecture when users have both central and local privacy costs: it is optimal to add noise locally!
Adding a noise centrally to the final estimator has an advantage because the weights in the final estimator give the platform a lever to deliver heterogeneous central privacy guarantees to users.
Despite this advantage, we establish that adding noise centrally is never optimal.
This is because the platform prefers to add the noise locally to contribute to both central and local privacy guarantees delivered to users.
4.2 Computing the optimal privacy loss function
The implementation of the optimal two-part private data acquisition mechanism requires solving
Problem (7), which is a non-convex program.
However, we use the structure of the problem to develop a polynomial time algorithm to solve it approximately.
To do so, we first replace (cid:80) ni
=1 w
i /y
i by an auxiliary variable S .
Next, we consider the corresponding lagrangian problem.
Using Karush-KuhnTucker (KKT) conditions, we establish a number of relations between problems’ parameters, S , and p , the lagrangian coefficient corresponding to S
= (cid:80) ni
=1 w
i /y i .
Furthermore, we develop upper and lower bounds for S .
Finally, we do a grid search to find the approximate optimal solution.
Theorem 3.
For any vector of reported privacy sensitivities and ϵ > 0 , Algorithm 1 finds local privacy loss levels and the differentially private linear estimator of the two-part data acquisition mechanism whose cost (i.e., platform’s objective) is at most 1 + δ of the optimal cost in time poly( n, 1 δ ) .
Notice that the approximation factor in Theorem 3 depends on the underlying parameters.
Therefore, we have a Polynomial Time Approximation Scheme (PTAS) for finding the optimal two-part data acquisition mechanism in the class of linear estimators.
In the Appendix, we provide a case study with two users to illustrate the performance of the optimal two-part data acquisition mechanism in terms of the guaranteed privacy levels and payments as functions of the reported privacy sensitivities.
Conclusion
In this paper we develop a unified framework to study the design of data acquisition mechanisms when users have both local and central privacy concerns and are heterogeneous in how they value these two privacy concerns.
We use Rényi differential privacy to measure the privacy loss of users and first establish a minimax lower bound that motivates us to focus on linear estimators.
We then establish a point-wise optimization problem whose solution fully characterizes the optimal data acquisition mechanism that constitute a payment scheme to compensate users for their privacy losses, a local privacy guarantee, and a central privacy guarantee all as a function of users’ preferences for local and central privacy concerns.
We then focus on linear estimators, motivated by our optimality results, and establish that, even though the corresponding optimization problem is non-convex, the platform’s problem admits a Polynomial Time Approximation Scheme.
Finally, we focused on data acquisition to estimate mean population.
However, our framework is more general and allows for considering other (potentially vector) estimates.
In particular, our Theorem 2 converts the data acquisition mechanism design problem into a (potentially) non-convex optimization problem.
Acknowledgement
Alireza Fallah acknowledges support from the Apple Scholars in Ai/ML PhD fellowship.