Bridging Central and Local Differential Privacy in Data Acquisition Mechanisms
Anonymous Author(s)
AffiliationAddress
Abstract
We study the design of optimal Bayesian data acquisition mechanisms for a platform interested in estimating the mean of a distribution by collecting data fromprivacy-conscious users.
In our setting, users have heterogeneous sensitivities fortwo types of privacy losses corresponding to local and central differential privacymeasures.
The local privacy loss is due to the leakage of a user’s information whenshe shares her data with the platform, and the central privacy loss is due to the released estimate by the platform to the public.
The users share their data in exchangefor a payment (e.g., through monetary transfers or services) that compensates fortheir privacy losses.
The platform does not know the privacy sensitivity of users andmust design a mechanism to solicit their preferences and then deliver both local andcentral privacy guarantees while minimizing the estimation error plus the expectedpayment to users.
We first establish minimax lower bounds for the estimation error,given a vector of privacy guarantees, and show that a linear estimator is (near)optimal.
We then turn to our main goal: designing an optimal data acquisitionmechanism.
We establish that the design of such mechanisms in a Bayesian setting(where the platform knows the distribution of users’ sensitivities and not theirrealizations) can be cast as a nonconvex optimization problem.
Additionally, forthe class of linear estimators, we prove that finding the optimal mechanism admitsa Polynomial Time Approximation Scheme.
Introduction
The data of billions of users are currently being utilized for personalized advertising, medical trials,targeted advertising and recommendation systems, among others.
The transaction of individual datais set to grow exponentially in the coming years, with more widespread applications of artificialintelligence (AI) and machine learning techniques.
Even though it is widely accepted that users needto own their data (see, e.g., Posner and Weyl [2019], Kushmaro [2021] and WILL.I.AM [2019]),the impact of different market architectures on the design and operation of data markets are notclear: some users prefer to protect their own raw data while others expect companies to protect theirdata proactively (see, e.g., GDMA [2018]).
In this paper, we consider the design of data acquisitionmechanisms when users have heterogeneous privacy concerns and ask the following question:
What is the optimal data acquisition mechanism when users have heterogeneousprivacy concerns regarding access to their raw data and the outcome of theplatform’s processing?
We use differential privacy to measure the two types of privacy losses.
Informally, an estimator iscalled differentially private if its distribution over outputs is insensitive to the changes in a user’s data.
In particular, we consider a platform whose goal is to estimate an underlying parameter of interestby collecting data from a set of users N = { 1 , . . . , n } who own a noisy version of the underlyingparameter.
For instance, consider a medical trial in which a hospital wants to collect users’ data toestimate the efficacy of a drug.
Each user has two types of privacy concerns: (i) local privacy concernthat captures how much information their shared data reveal about their raw data, and (ii) centralprivacy concern that captures how much information the platform’s output reveal about their rawdata.
We adopt local and central Rényi differential privacy to measure these two types of privacylosses.
The reason for choosing Rényi differential privacy is twofold.
First, our framework can covera wide range of information measures by varying the Rényi divergence parameter.
Second, it can beachieved by a Gaussian mechanism, which simplifies our analysis while capturing the main tradeoffsin the design of two-part data acquisition mechanisms.
Before formulating the platform’s data acquisition problem, we derive optimal estimators for a givenvector of heterogeneous local privacy loss levels.
In particular, we establish a minimax lower boundfor the estimation error and prove that first privatizing users’ data by adding a properly designed
Gaussian noise to them and then using a properly designed weighted average of these privatized datapoints achieves this lower bound.
This result motivates us to consider the design of the optimal dataacquisition mechanism for the class of linear estimators.
We then turn to our mechanism design problem.
Each user has a heterogeneous preference regardingthe importance of the above two privacy concerns.
For instance, if a user fully trusts the platform,then the first type of concern lessens, and the main concern would be about the information revealedfrom the platform’s estimate.
On the other hand, if a user does not trust the platform at all, the firsttype of concern would be the more the second one.
We model such a setting by assuming each user ihas a privacy sensitivity c i ∈ [0 , 1] that determines the relative weight she puts on the local privacyconcern ( 1 − c
i is the weight she puts on the central privacy concern).
The utility of user i is thepayment she receives from the platform (in exchange for sharing her data), minus c
i times her localprivacy loss, and again, minus 1 − c i times her central privacy loss.
The platform does not know thevalue of c i and (knowing its distribution) must design a (Bayesian) data acquisition mechanism toelicit the true privacy sensitivities (that guide the optimal choice of local and central privacy lossesdelivered to each user) and optimize its objective.
In particular, the platform designs a two-part data acquisition mechanism that comprises a paymentscheme, a local privacy guarantee, and a central privacy guarantee as a function of the reported privacysensitivity of users.
The platform’s problem is to minimize the sum of the mean estimation errorand the expected total payment to users while satisfying the incentive compatibility and individualrationality constraints.
Incentive compatibility ensures that users have no incentive to lie about theirprivacy sensitivity and report them truthfully, and individual rationality ensures that the payment tousers (and the delivered privacy guarantees) are such that users are willing to share their data with theplatform.
The platform’s problem is a functional optimization over three functions of the reported privacysensitivities: payments, local privacy guarantees, and central privacy guarantees.
We first findthe payment function in terms of the local and central privacy guarantees by using the incentivecompatibility and individual rationality constraints.
This reduces the space of the platform’s decisionvariables.
We then focus on the Gaussian mechanisms and linear estimators (motivated by ourminimax optimality result) and show that the platform’s problem can be cast as an optimizationproblem that minimizes a non-convex objective (which depends on the virtual cost of users)
forany reported vector of privacy sensitivities.
This reformulation significantly reduces the space ofdecision variables that the platform needs to optimize.
However, it still involves solving a non-convexoptimization problem.
We further use the structural properties of this non-convex optimization anduse duality theory to develop a polynomial time algorithm to approximate the platform’s problem.
More precisely, we prove that the design of the optimal two-part data acquisition mechanism admitsa Polynomial Time Approximation Scheme (PTAS).
The contribution of our work is threefold.
First, we develop a minimax lower bound when users haveheterogeneous local privacy losses and establish that a linear estimator (approximately) achieves thisbound.
Second, We develop a modeling framework for data acquisition mechanisms when usershave heterogeneous concerns for both local and central privacy losses.
Third, for any estimator andmechanism to deliver privacy guarantees, we characterize the design of the optimal two-part dataacquisition mechanism as the solution to a point-wise optimization problem.
Additionally, for the class of Gaussian mechanisms to deliver privacy guarantees and linear estimators, we develop analgorithm to approximately find the optimal data acquisition mechanism (despite the fact that thecorresponding optimization is non-convex).
1.1 Related literature
Our paper relates to the literature on optimal data acquisition from privacy concerned users.
Thereis a large body of work that use differential privacy to measure the privacy loss of users Ghosh and
Roth [2011], Nissim et al.
[2012], Nissim et al.
One of the earliest papers in this literatureis Ghosh and Roth [2011], which study the design of mechanisms for collecting users’ data whenusers incur some privacy cost from sharing their data.
More specifically, Ghosh and Roth [2011]consider binary data (bit) with the platform’s goal being to estimate the sum of user’s data by using adifferentially private and dominant strategy truthful mechanism.
They study both the case when theuser data and privacy parameter are independent (similar to our paper) and when they are correlated.
In the independent case, they propose a mechanism that delivers a single privacy level to all users (asopposed to our setting that delivers heterogeneous privacy levels).
For the correlated case, they provean impossibility result for the existence of a truthful and individually rational mechanism.
Several works build on Ghosh and Roth [2011], extending it to take it or leave it offers Ligett and Roth[2012], strengthening the impossibility results Nissim et al.
[2014], and studying the open questionposed by Ghosh and Roth [2011] on whether a model with distributional assumption on users’ costsand Bayesian mechanism design approach could be used to develop the optimal mechanism forcollecting data with privacy guarantees (see, e.g., Liao et al.
[2021] and Fallah et al.
Inparticular, Roth and Schoenebeck [2012], Chen et al.
[2018], and Chen and Zheng [2019] tackle thisproblem by developing a randomized mechanism in which user’s data is randomly included in thefinal estimator where the inclusion probability depends on the reported privacy costs of the users(as opposed to our setting in which the payments and privacy guarantees depend on the reportedprivacy sensitivity of all users).
These papers do not use differential privacy to model privacy costsand instead use a menu of probability-price pairs to tune the privacy loss and the payment to each user(see also Pai and Roth [2013] for a survey).
Similar to the above paper, we consider a setting in whichthe platform can verify the data of users.
For instance, in the context of medical trials, this means thatthe users decide whether to participate in the medical trial and cannot change the samples they share.
A different stream of this literature explores settings in which users can misreport their information
Perote and Perote-Pena [2003], Dekel et al.
[2010], Meir et al.
[2012], Ghosh et al.
[2014], Cai et al.[2015], Liu and Chen [2016, 2017].
Our paper differs from these works in three main ways.
First, we assume prior information onuser privacy sensitivities and focus on characterizing the optimal Bayesian incentive compatiblemechanism.
Second, we model a setting in which users have both local and central privacy concernsand explore the different privacy guarantees of these two types delivered by an optimal mechanism.
Third, we assume that user data are drawn from the same underlying distribution that allows theplatform to put differing weights on the data of users depending on their privacy sensitivity, leadingto different privacy levels for participating users.
Finally, our paper relates to the literature on differential privacy.
Pioneered by the seminal work of
Dwork et al.
[2006a,b], differential privacy has emerged as a prevalent framework for characterizingthe privacy leakage of data oriented algorithms.
More specifically, our paper is related to the privatemean estimation considered by Duchi et al.
[2013], Barber and Duchi [2014], Karwa and Vadhan[2017], Asoodeh et al.
[2021], Kamath et al.
[2019, 2020], Cummings et al.
[2021], and Acharyaet al.
Additionally, our paper relates to the stream of differential privacy literature that studies
Rényi differential privacy (RDP) introduced by Bun and Steinke [2016] and Mironov [2017].
Problem Formulation
We consider a platform interested in estimating a parameter θ ∈ R by collecting data of n users,indexed by N = { 1 , · · · , n } .
For any i ∈ N , we denote user
i ’s data by x i ∈ X
and we assumex i is given by x i = θ
+ z i where z 1 , · · · , z n are independent and identically distributed zero-meanrandom variables with variance VAR .
To simplify the exposition, we further assume | Z i | ≤ 1 / 2 forany i ∈ N .
The platform’s goal is to minimize the estimate’s error by collecting data from users.
The users are concerned about their privacy losses and therefore the platform needs to incentivizethem to share their data.
2.1 Local and central privacy losses
Before formalizing the utilities/objectives of the platform and the users, we define the notions ofprivacy losses that we adopt in this paper.
In particular, we consider two different types of privacylosses that users suffer from.
The first one is the privacy loss of a user when she shares her data (only)with the platform, and the second one is the privacy loss through the released estimate (to the public)by the platform.
Depending on how different users trust the platform, they might care differentlyabout these two privacy losses.
For instance, if a user fully trusts the platform, then her main privacyconcern would be the second one, while a user who does not trust the platform at all would be moreconcerned with the first one as the public only observes the aggregated estimate, as opposed to theplatform which observes each user’s (shared) data separately.
We use the differential privacy framework to quantify these privacy losses.
Since differential privacywas introduced by Dwork et al., several variants of it have been also proposed.
In particular, a popularone in the machine learning literature is Rényi differential privacy (RDP), introduced by Mironov[2017], which we also adopt in this paper.
To state the RDP’s definition, we first need to recall thedefinition of Rényi divergence.
Definition 1.
Let P and Q be two distributions over R with densities p and q .
For any α ∈ (1 , ∞ ] ,the Rényi α -divergence between P and Q is denoted by D α ( P || Q ) and is given by
[Equation]
Furthermore, for two random variables X and Y , D α ( X || Y ) denotes the α -divergence betweentheir distributions.
We next define two notions of differential privacy, known as central and local, to capture the twoaforementioned types of privacy losses.
More specifically, local differential privacy corresponds tothe privacy loss of a user when she shares her data with the platform through a randomized mapping,known as a channel .
Definition 2.
Let ε ≥ 0 and α ∈ (1 , ∞ ] .
A randomized channel C : X → R is locally ( ε, α ) -Rényi(differentially) private if for any x, x ′ ∈ X ,
[Equation]
Central differential privacy corresponds to the other privacy loss mentioned above.
It bounds thechange in the distribution of the platform’s output, i.e., the released estimate, by changing one user’sdata.
We next provide the formal definition.
Definition 3.
Let ε
=
( ε i ) ni
=1 ∈ R n + and α ∈ (1 , ∞ ] .
A randomized algorithm A : X n → R is( ε , α ) -Rényi (differentially) private if for any two datasets x 1: n , x ′ 1: n ∈ X n that only differ in i -thcoordinate (data of user i ),
[Equation]
The customary approach to guarantee RDP is Gaussian mechanism in which a properly tuned zeromean Gaussian noise is added to fulfill the required condition.
The following lemma, adapted from
Mironov [2017], allows us to characterize the Gaussian noise’s variance for a certain privacy losslevel.
Lemma 1.
Let α ∈ (1 , ∞ ] .
For a function f : X n → R , we define its sensitivity with respect to i -thcoordinate as
L i ( f ) :
= sup {| f ( x 1: n ) − f ( x ′ 1: n ) | : for all x 1: n and x ′ 1: n differing only at i -th coordinate } .
Then, the mechanism A ( x 1: n ) =
f ( x 1: n ) + W with W ∼ N (0 , σ 2 ) is (cid:16) ( αL i ( f ) 2 2 σ 2 ) ni
=1 , α (cid:17) -Rényidifferentially private.
For a given vector of local privacy losses ( ε ( l ) 1 , . . .
, ε ( l ) n ) , a natural way to privately estimate the meanis by using a linear estimator with Gaussian mechanism which is given byˆ θ ( x 1 , . . .
, x n ) := n (cid:88) i =
w i ˆ x i where n (cid:88)
i =
w i = 1 and ˆ x i
= x
i + N (cid:32) 0 , α 2 ε ( l ) i (cid:33) for all i ∈ N .
Figure 1 depicts this estimator.
In particular, using Lemma 1, one can verify that the local privacydelivered to user i ∈ N is ε ( l ) i and the central privacy delivered to user i ∈ N is ε ( c ) i
[Equation]
In the next section, we focus on linear estimators in designing the optimal private data acquisitionmechanism.
We motivate such specification by showing that, for a given vector of local privacylosses ( ε ( l ) i ) ni =1 , a linear estimator is optimal with respect to mean square error.
To formalize thisstatement, we first need to define the minimax estimation error as the notion of optimality.
Let P be aclass of distributions over X .
For any P ∈ P , we denote its mean by θ ( P ) .
A ( ε ( l ) i ) ni
=1 -locally RDPestimator can be cast as ˆ θ (( C i ( x i ))
ni =1 ) , where C i ( . ) is the randomized channel corresponding to useri .
Let Q (( ε ( l ) i ) ni =1 ) be the class of such ( ε ( l ) i ) ni =1 -locally RDP estimators.
The minimax estimationerror is given by
[Equation] P [Equation] Q ,
[Equation]
[Equation]
where the expectation is taken over both randomness of data and estimator (including private channels).
In other words, the optimal estimator is the one that has the lowest worst case error among allestimators that satisfy the privacy requirements.
We next state the optimality result.
Theorem 1.
Assume α ≥ 2 and ε ( l ) i ≤ 1 for all i .
Let P 1 be the family of distributions over[ − 1 / 2 , 1 / 2] . Suppose C 1 , · · · , C n are independent channels.
Then, there exists a universal constant csuch that
[Equation]
Furthermore, there exists a linear estimator with Gaussian mechanism such that
[Equation]
We prove the lower bound by using the Le Cam’s method Yu [1997] that reduces the problem offinding lower bounds to a hypothesis testing problem and requires bounding the total variationdistance between the estimates when the input data points change.
We then find it more convenient tobound the total variation in terms of Hellinger distance and use a series of inequalities to bound it.
2.2 Data acquisition mechanism with two-part privacy guarantees
We next describe the utility functions of the users and the platform and then formulate the platform’soptimal data acquisition mechanism.
As we described earlier, each user suffers from two privacylosses when sharing her data.
The first one is a central privacy loss because of the leakage of herinformation through the platform’s output.
The second one is a local privacy loss because of theleakage of her information through the raw data that she shares with the platform.
Each user has aheterogeneous privacy sensitivity for these two types of privacy losses.
To model such heterogeneity,for each i ∈ N , we let c i ∈
[0 , 1] be her relative local privacy sensitivity , representing the relativeweight that user i assigns to the (per unit cost of) local privacy loss.
We also let 1 − c
i be her relativecentral privacy sensitivity , representing the relative weight that user i assigns to the (per unit cost of)central privacy loss.
Therefore, c i ≈ 1 implies that user i suffers a higher loss of privacy by sharing her raw data with the platform (local privacy loss) compared to her loss from the platform’s output(central privacy loss).
Differently, c i ≈ 0 implies that the user suffers a smaller loss of privacy bysharing her raw data with the platform compared to her loss from the platform’s output.
In whatfollows, we use the term privacy sensitivity instead of relative local privacy sensitivity.
For each i ∈ N , the privacy sensitivity c i is independently drawn from a publicly known distributionwhose support is [0 , 1] with cumulative distribution and probability density functions F i ( · ) and f i ( · ) .
We also let c = ( c 1 , . . .
, c n ) be the vector of privacy sensitivities.
The privacy sensitivity of eachuser is her private information, i.e., the platform does not know it.
This is because individuals havedifferent views regarding how trustworthy the platform is in protecting their raw data.
The platform’s objective is to design a mechanism to collect users’ data by paying them to compensatefor their privacy losses without knowing the privacy sensitivity of users.
To introduce the platform’sobjective formally, we adopt the formalism of Bayesian mechanism design pioneered by Myerson[1981].
More specifically, the platform designs and announces a payment function, a local privacy lossfunction, and a central privacy loss function that are mappings from the reported privacy sensitivitiesof users.
The users then report their privacy sensitivities (which may or may not be truthful).
Basedon the payment function, the platform compensates the users (the compensation could be monetaryor some free or discounted service provided to the user).
Based on the local and central privacyfunctions, the platform designs randomized channels and randomized estimation algorithms thatdeliver the guaranteed local and central privacy losses while minimizing the sum of the mean squarederror and the total expected payments.
Given this interaction, we next formally introduce a dataacquisition mechanism with two-part data privacy guarantees.
Definition 4 (two-part private data acquisition mechanism) .
We call the tuple (ˆ θ, ε ( l ) , ε ( c ) , t ) atwo-part private data acquisition mechanism where
For all i ∈ N , ε ( l ) i : X → ˆ X is a function that maps the vector of privacy sensitivities c to alocal privacy loss for user i , ε ( l ) i ( c ) , with ε ( l ) ( · ) =
( ε ( l ) i ( · ))
ni =1 .
For all i ∈ N , ε ( c ) i : R n + → R + is a function that maps the vector of privacy sensitivities cto a central privacy loss for user i , ε ( c ) i ( c ) , with ε ( c ) ( . )
= ( ε ( c ) i ( · ))
ni =1 .3.
ˆ θ : X n × R n + × R n + → R is a ( ε ( c ) ( c ) , α ) -Rényi differentially private estimator that mapsacquired locally ( ε ( l ) i ( c ) , α ) -Rényi differentially private data of user i for i ∈ N to anestimate ˆ θ ( x , ε ( l ) ( c ) , ε ( c ) ( c )) .4.
For all i ∈ N , t i : R n + → R + is a function that maps the vector of privacy sensitivities c toa payment for user i , t i ( c ) , with t ( . )
=
( t i ( · )) ni =1 .
Notice that we have not specified the estimator and the mechanisms that delivers (local and central)
Rényi differential privacy.
In the rest of this subsection, we introduce the utilities and the platform’sproblem for a general estimators and mechanisms to deliver differential privacy.
Later, we focus onlinear estimator and Gaussian mechanisms and explicitly solve the platform’s problem.
Each user that participates in a two-part private data acquisition mechanism suffers from both thelocal and central privacy losses and need to be compensated by the platform.
In particular, the utilityof user i from participation when her privacy sensitivity is c
i
and she reports c
i is given by
[Equation]
t i ( c − i , c ′ i ))
− c i ε ( l ) i
( c − i , c ′ i ) − (1 − c i )
ε ( c ) i
c − i , c ′ i ) (cid:105) , where the term t i ( c − i , c ′ i )) is the payment from the platform, the term c i ε ( l ) i ( c − i , c ′ i ) is the relative local privacy sensitivity of the user multiplied by her local privacy loss, and the term (1 − c i ) ε ( c ) i ( c − i , c ′ i )is her relative central privacy sensitivity multiplied by her central privacy loss.
A user i ∈ N thatdoes not participate in the mechanism neither compromises her privacy nor gets a compensation.
Therefore, the utility of a user who does not participate in the mechanism becomes 0 .
The goal of the platform is to minimize the sum of the mean squared error and the overall payment tousers.
We let γ ∈ R + represents the relative weight of the mean estimation error and the payments in the platform’s objective.
Therefore, the platform’s objective is
[Equation]
where the first term is the mean square error of estimator ˆ θ given reported vector of privacy sensitivityand resulting local and central privacy losses ε ( l ) and ε ( c ) , i.e.,
[Equation]
Also, each summand of the second term is the compensation that the platform gives to a user toincentivize her to participate and report her privacy sensitivity truthfully.
In Appendix we prove that, similar to the classical mechanism design setting, revelation principleholds.
This means that there is no loss of generality in focusing on the class of direct incentivecompatible mechanisms, meaning the platform’s optimization problem can be written as
[Equation]
where the constraints in (4) represent the incentive compatibility .
These constraints guarantee thatthat each user i has no incentive to misrepresent her privacy sensitivity when others report truthfully(reporting truthfully is an equilibrium of the game among the users).
Also, the constraints in (5)represent individual rationality , which ensures that each user receives a non-negative utility fromparticipating in the platform’s mechanism and sharing her data.
From the mechanism design problem to an optimization problem
For a given estimator ˆ θ , the platform’s decision comprises the local and central privacy loss functionsε ( l ) ( · ) and ε ( c ) ( · ) together with the payment functions t ( · ) .
We next show that this problem can beequivalently formulated as an optimization problem over the vector of local privacy losses and centralprivacy losses (as opposed to functions).
In the rest of the paper, we impose the following assumptionwhich is well-known in the mechanism design literature and simplifies the analysis.
Assumption 1.
For any user i ∈ N , the virtual cost defined as ψ i ( c ) =
c
+ F i ( c ) f i ( c ) is increasing in c .
The above assumption holds for a wide class of distributions such as the ones with log-concavedensity functions (e.g., uniform).
Theorem 2.
Suppose Assumption 1 holds.
For a given estimator ˆ θ : ˆ X n × R n + × R n + → R , in theoptimal two-part data acquisition mechanism, for a given vector of reported privacy sensitivities c ,the local and central privacy losses are the solution of
[Equation]
Proof Sketch of Theorem 2: We introduce the following interim functions
[Equation]
and
[Equation]
We first establish a payment identity that determines the optimal payment in terms of the optimallocal and central privacy losses.
In particular, by evaluating the first order condition corresponding tothe incentive compatibility constraint (4), we establish that this constraint holds if and only if
[Equation]
Without this assumption, extending the results requires ironing technique of Myerson [1981].
and ε ( l ) i
( z ) − ε ( c ) i
( z ) is weakly decreasing in z .
We then plug in this payment identity back to theplatform’s objective, use the individual rationality constraint, and rewrite the platform’s expectedutility in terms of the privacy loss functions and the virtual cost of users.
This problem is still afunctional optimization in terms of ε ( l ) ( · ) and ε ( c ) ( · ) .
However, we establish that, under Assumption1, we can solve this functional optimization point-wise (i.e., for any given c ).
■
Theorem 2 highlights the tradeoff in the platform’s problem: by decreasing the local privacy loss,the second term of the objective decreases (this term corresponds to the payment to users)
whilethe first term (i.e., the mean squared error) increases.
The role of the central privacy loss is morenuanced, and there are two cases.
If the coefficient 1 − ψ i ( c i ) is non-negative, by decreasing thecentral privacy loss, the third term of the objective decreases while the first term increases.
If thecoefficient 1 − ψ i ( c i ) is negative, increasing the central privacy loss decreases both the third termand the first term.
However, we cannot increase the central privacy loss level without limits becausethe central privacy loss level is always below the local privacy loss level.
Therefore, the platform’soptimal mechanism should find the “right” balance between these terms.
Optimal mechanism with two-part privacy guarantees for linear estimators
For the rest of the paper, we focus on linear estimators with Gaussian mechanism described in Section2.1.
The following is a direct corollary of Theorem 2.
Corollary 1.
Suppose Assumption 1 holds.
For any reported vector of privacy sensitivities( c 1 , . . . , c n ) , the optimal local privacy loss levels are ε ( l ) i ( c ) =
y ∗ i for i ∈ N , the optimal central privacy loss levels are
[Equation]
where ( w ∗ 1 , . .
w ∗ n ) and ( y ∗ 1 , . . . , y ∗ n ) are the optimal solution of min w , y VAR γ n
[Equation]
s
[Equation]
w i , y i ≥ 0 , for all
i ∈ N and
[Equation]
Let us highlight the difference between our characterization and that of classic mechanism design (e.g.,
Myerson [1981]).
In classic mechanism design, the designer’s problem becomes linear optimization.
However, in our setting, the designer’s problem is a non-linear and non-convex optimization.
Thismakes the problem of finding the optimal two-part data acquisition mechanism challenging.
Beforeaddressing this computational challenge, let us revisit the form of the Gaussian mechanism that wehave adopted: the platform adds Gaussian noise locally and then outputs a convex combination ofthe privatized users’ data without adding any noise centrally.
More specifically, one may guess thatthe platform may benefit by having a central noise added to the final output in addition to the localnoises.
In the following subsection, we establish that there is another Gaussian mechanism for any
Gaussian mechanism that only adds local noises and achieves a weakly lower cost.
4.1 Optimality of having only local noises in the Gaussian mechanism
The platform has the opportunity of adding Gaussian noise to both the raw data of each user and thefinal estimator and ex-ante one may guess that it is optimal to use both of these instruments.
However,as we establish next, interestingly, in the optimal two-part data acquisition mechanism, it is alwaysoptimal to only add noises locally.
For a given vector of local privacy losses ( ε ( l ) 1 , . . .
, ε ( l ) n ) and central privacy losses ( ε ( c ) 1 , . .
, ε ( c ) n ) , a
Gaussian mechanism with both local and central noises is of the formˆ θ ( x 1 , . . .
, x n ) := n (cid:88) i = w i ˆ x i + N (cid:16) 0 , α 2 ε (cid:17) where n (cid:88) i =
w i = [-
-]1 and ˆ x i
= x i + N (cid:32) 0 , α 2 ε ( l ) i (cid:33) ∀ i ∈ N .
Proposition 1.
In the optimal two-part data acquisition mechanism that adopts a Gaussian mechanism with both local and central noises, we have ε = ∞ .
do do
For all i ∈ N , let w i ( S, p )
=
ν i
+ ν ν j √ ψ j ( c j ) p (cid:80) j ν j −
ν
i (cid:112) ψ i ( c i ) p , ν j = 1 γ VAR +(1 − ψ j ( c j )) /S , y i ( S, p ) =
w i ( S, p ) (cid:113) p ψ i ( c i ) , and OBJ( S, p ) be the objective of Problem (7) evaluated for this solution.
end Proposition 1 has an important implication in terms of the design of data market architecture whenusers have both central and local privacy costs: it is optimal to add noise locally!
Adding a noisecentrally to the final estimator has an advantage because the weights in the final estimator give theplatform a lever to deliver heterogeneous central privacy guarantees to users.
Despite this advantage,we establish that adding noise centrally is never optimal.
This is because the platform prefers to addthe noise locally to contribute to both central and local privacy guarantees delivered to users.
4.2 Computing the optimal privacy loss function
The implementation of the optimal two-part private data acquisition mechanism requires solving
Problem (7), which is a non-convex program.
However, we use the structure of the problem to developa polynomial time algorithm to solve it approximately.
To do so, we first replace (cid:80) ni
=1 w
i /y
i by anauxiliary variable S .
Next, we consider the corresponding lagrangian problem.
Using Karush-KuhnTucker (KKT) conditions, we establish a number of relations between problems’ parameters, S , andp , the lagrangian coefficient corresponding to S
= (cid:80) ni
=1 w
i /y i .
Furthermore, we develop upper andlower bounds for S and p .
Finally, we do a grid search to find the approximate optimal solution.
Theorem 3.
For any vector of reported privacy sensitivities ( c 1 , . . . , c n ) and ϵ > 0 , Algorithm 1finds local privacy loss levels and the differentially private linear estimator of the two-part dataacquisition mechanism whose cost (i.e., the platform’s objective) is at most 1 + δ of the optimal costin time poly( n, 1 δ ) .
Notice that the approximation factor in Theorem 3 depends on the underlying parameters.
Therefore,we have a Polynomial Time Approximation Scheme (PTAS) for finding the optimal two-part dataacquisition mechanism in the class of linear estimators.
In the Appendix, we provide a case study with two users to illustrate the performance of the optimaltwo-part data acquisition mechanism in terms of the guaranteed privacy levels and payments asfunctions of the reported privacy sensitivities.
Conclusion
In this paper we develop a unified framework to study the design of data acquisition mechanisms whenusers have both local and central privacy concerns and are heterogeneous in how they value these twoprivacy concerns.
We use Rényi differential privacy to measure the privacy loss of users and firstestablish a minimax lower bound that motivates us to focus on linear estimators.
We then establisha point-wise optimization problem hose solution fully characterizes the optimal data acquisitionmechanism that constitute a payment scheme to compensate users for their privacy losses, a localprivacy guarantee, and a central privacy guarantee all as a function of users’ preferences for local andcentral privacy concerns.
We then focus on linear estimators, motivated by our optimality results, andestablish that, even though the corresponding optimization problem is non-convex, the platform’sproblem admits a Polynomial Time Approximation Scheme.
A shortcoming of our analysis is that wederive the lower bound for a given vector of local privacy losses (and not for the mechanism designproblem itself).
Studying the optimality of linear estimators for the mechanism design problemremains an open problem.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes] Please check the conclusionsection.(c)
Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Please check Sections 2,3, and 4.
(b) Did you include complete proofs of all theoretical results?
[Yes] Please check appendix
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A
c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A