Title: Incentives in Private Collaborative Machine Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Incentives

Privacy

Shapley fairness

Collaborative machine learning

data valuation

reward

sufficient statistics
Abstract: Collaborative machine learning involves training models on data from multiple parties but must incentivize their participation. Existing data valuation methods fairly value and reward each party based on  shared data or model parameters but neglect the privacy risks involved. To address this, we introduce _differential privacy_ (DP) as an incentive. Each party can select its required DP guarantee and perturb its _sufficient statistic_ (SS) accordingly. The mediator values the perturbed SS by the Bayesian surprise it elicits about the model parameters. As our valuation function enforces a _privacy-valuation trade-off_, parties are deterred from selecting excessive DP guarantees that reduce the utility of the grand coalition's model. Finally, the mediator rewards each party with different posterior samples of the model parameters. Such rewards still satisfy existing incentives like fairness but additionally preserve DP and a high similarity to the grand coalition's posterior. We empirically demonstrate the effectiveness and practicality of our approach on synthetic and real-world datasets.

Incentives in Private Collaborative Machine Learning

Abstract

Collaborative machine learning involves training models on data from multiple parties but must incentivize their participation. Existing data valuation methods fairly value and reward each party based on shared data or model parameters but neglect the privacy risks involved. To address this, we introduce differential privacy (DP) as an incentive. Each party can select its required DP guarantee and perturb its sufficient statistic (SS) accordingly. The mediator values the perturbed SS by the Bayesian surprise it elicits about the model parameters. As our valuation function enforces a privacy-valuation trade-off , parties are deterred from selecting excessive DP guarantees that reduce the utility of the grand coalition's model. Finally, the mediator rewards each party with different posterior samples of the model parameters. Such rewards still satisfy existing incentives like fairness but additionally preserve DP and a high similarity to the grand coalition's posterior. We empirically demonstrate the effectiveness and practicality of our approach on synthetic and real-world datasets.

1 Introduction

Collaborative machine learning (ML) seeks to build ML models of higher quality by training on more data owned by multiple parties [47, 62]. For example, a hospital can improve its prediction of disease progression by training on data collected from more and diversified patients from other hospitals [6]. Likewise, a real-estate firm can improve its prediction of demand and price by training on data from others [9]. However, parties have two main concerns that discourage data sharing and participation in collaborative ML: (a) whether they benefit from the collaboration and (b) privacy.

Concern (a) arises as each party would expect the significant cost that it incurs to collect and share data (e.g., the risk of losing its competitive edge) to be covered. Some existing works [47, 51], among other data valuation methods, 1 have recognized that parties require incentives to collaborate, such as a guaranteed fair higher reward from contributing more valuable data than the others, an individually rational higher reward from collaboration than in solitude, and a higher total reward (i.e., group welfare ) whenever possible. Often, parties share and are rewarded with information (e.g., gradients [58] or parameters [47] of parametric ML models) computed from the shared data. However, these incentive-aware reward schemes expose parties to privacy risks.

On the other hand, some federated learning (FL) works [34] have addressed the privacy concern (b) and satisfied strict data protection laws (e.g., European Union's General Data Protection Regulation) by enforcing differential privacy (DP) [1, 36] during the collaboration. Each party injects noise before sharing information to ensure that its shared information would not significantly alter a knowledgeable collaborating party's or mediator's belief about whether a datum was input to the algorithm. Injecting more noise leads to a stronger DP guarantee. As raised in [64], adding DP can invalidate game-theoretic properties and hence affect participation. For example, in the next paragraph, we will see that adding DP may lead to the collaboration being perceived as unfair and a lower group welfare. However, to the best of our knowledge (and as discussed in Sec. 7 and Fig. 5), there are no works that address both concerns, i.e., ensure the fairness , individual rationality , and group welfare incentives (see Sec. 4), alongside privacy. Thus, we aim to fill in this gap and design an incentive-aware yet privacy-preserving reward scheme by addressing the following questions:

If a party (e.g., hospital) requires a stronger DP guarantee, what should the impact be on its valuation and reward? Our answer is that, on average, its valuation and reward should decrease. Intuitively, it is unfair when this party gets a higher valuation due to randomness in the DP noise. More importantly, parties require guaranteed higher rewards to consider a weaker privacy guarantee [22, 64] which will help maximize the utility of the collaboratively trained model(s). As observed in [14, 65], the weaker the DP guarantee, the smaller the loss in model accuracy from enforcing DP. Thus, we will (i) assign a value to each party to enforce a privacy-valuation trade-off and incentivize parties against unfetteredly selecting an excessively strong DP guarantee, 2 and (ii) flexibly allow each party to enforce a different DP guarantee without imposing a party's need for strong DP on others. This new perspective and its realization is our main contribution.

To enforce a privacy-valuation trade-off, how should DP be ensured and a party's data be valued (Sec. 3)? Initially, valuation using validation accuracy seems promising as the works of [18, 25] have empirically shown that adding noise will decrease the valuation. However, parties may be reluctant to contribute validation data due to privacy concerns and disagree on the validation set as they prioritize accurate predictions on different inputs (e.g., patient demographics). So, we revert to valuing parties based on the quality of inference of the model parameters under DP. Bayesian inference is a natural choice as it quantifies the impact of (additional DP) noise. In Sec. 2, we will explain how each party ensures DP by only sharing perturbed sufficient statistic (SS) with the mediator. The mediator values the perturbed SS by the surprise it elicits relative to the prior belief of model parameters. Intuitively, noisier perturbed SS is less valuable as the posterior belief of the model parameters will be more diffuse and similar to the prior. As parties prioritize obtaining a model for future predictions and may face legal/decision difficulties in implementing monetary payments, we reward each party with posterior samples of the model parameters (in short, model reward ) instead.

How should the reward scheme be designed to satisfy the aforementioned privacy, individual rationality, and fairness incentives (Sec. 4)? Our scheme will naturally satisfy the privacy incentive as any post-processing of the perturbed SS will preserve DP. To satisfy fairness and individual rationality, we set the target reward value for every party using ρ -Shapley value [47]. Lastly, to realize these target reward values, how should the model reward be generated for each party (Sec. 5)? Instead of rewarding all parties with samples from the same (grand coalition's) posterior of the model parameters given all their perturbed SS (which would be unfair if their valuations differ), our reward control mechanism generates a different posterior for each party that still preserves a high similarity to the grand coalition's posterior. Concretely, the mediator scales the SS by a factor between 0 and 1 before sampling to control the impact of data on the posterior (by tempering the data likelihood). Scaling the SS by a factor of 0 , 1 , and between 0 and 1 yield the prior, posterior, and their interpolation, respectively. We then solve for the factor to achieve the target reward value.

By answering the above questions, our work here provides the following novel contributions 3 :

A new privacy-valuation trade-off criterion for valuation functions that is provably satisfied by the combination of our Bayesian surprise valuation function with DP noise-aware inference (Sec. 3);

New incentives including DP (while deterring excessive DP ) and similarity to grand coalition's model (Sec. 4);

Reward control mechanisms (Sec. 5) to generate posterior samples of the model parameters for each party that achieve a target reward value and the aforementioned incentives; one such mechanism tempers the likelihood of the data by scaling the SS and data quantity.

2 Collaborative ML Problem with Privacy Incentive

Our private collaborative ML problem setup comprises a mediator coordinating information sharing, valuation, and reward, and n parties performing a common ML task (e.g., predicting disease progression). Let the set N ≜ { 1 , . . . , n } denote the grand coalition of n parties. Each party i owns a private dataset D i which cannot be directly shared with others, including the mediator. What alternative information should each party provide to the mediator for collaborative training of an ML model?

To ease aggregation, this work focuses only on Bayesian models with sufficient statistic (SS), such as exponential family models [5], Bayesian linear regression [39], and generalized linear models, including Bayesian logistic regression [21] (with approximate SS).

Definition 2.1 (Sufficient Statistic (SS) [48, 52]) . The statistic s i is a SS for the dataset D i if the model parameters θ and dataset D i are conditionally independent given s i , i.e., p ( θ | s i , D i ) = p ( θ | s i ) .

We propose that each party i shares its SS s i for and in place of its dataset D i to protect the privacy of D i . We assume that the parties have agreed to adopt a common Bayesian model with the same prior p ( θ ) of model parameters θ , and each party i 's dataset D i is independently drawn from the likelihood p ( D i | θ ) that is conjugate to the prior p ( θ ) (i.e., belonging to an exponential family). The mediator can compute the posterior belief p ( θ |{D i } i ∈ N ) of model parameters θ given the grand coalition N 's datasets using a function f θ of the sum over shared SS: p ( θ |{D i } i ∈ N ) ∝ p ( θ ) f θ ( ∑ i ∈ N s i ) . We give a concrete example and the mathematical details of SS in Apps. A.1 and E, respectively.

Privacy Incentive. However, sharing the exact SS s N ≜ { s i } i ∈ N will not ensure privacy as the mediator can draw inferences about individual datum in the private datasets D N ≜ {D i } i ∈ N . To mitigate the privacy risk, each party i should choose its required privacy level ϵ i and enforce ( λ, ϵ i ) -Rényi differential privacy . 4 In Def. 2.2, a smaller ϵ i corresponds to a stronger DP guarantee. 4

Definition 2.2 (Rényi Differential Privacy (DP) [38]) . A randomized algorithm R : D → o is ( λ, ϵ ) -Rényi differentially private if for all neighboring datasets D and D ′ , the Rényi divergence 4 of order 4 λ > 1 is D λ ( R ( D ) || R ( D ′ )) ≤ ϵ .

Party i can enforce (example-level) 4 ( λ, ϵ i ) -Rényi DP by applying the Gaussian mechanism: It generates perturbed SS o i ≜ s i + z i by sampling a Gaussian noise vector z i from the distribution p ( Z i ) = N ( 0 , 0 . 5 ( λ/ϵ i ) ∆ 2 2 ( g ) I ) where ∆ 2 2 ( g ) is the squared ℓ 2 -sensitivity 4 of the function g that maps the dataset D i to the SS s i . We choose Rényi DP over the commonly used ( ϵ, δ ) -DP as it gives a stronger privacy definition and allows a more convenient composition of the Gaussian mechanisms [38], as explained in App. A.2.

Each party i will share (i) the number c i ≜ |D i | of data points in its dataset D i , (ii) its perturbed SS o i , 5 and (iii) its Gaussian distribution p ( Z i ) with the mediator. As DP algorithms are robust to post-processing, the mediator's subsequent operations of o i (with no further access to the dataset) will preserve the same DP guarantees. The mediator uses such information to quantify the impact of the DP noise and compute the DP noise-aware posterior 6 p ( θ |{ o i } i ∈ N ) via Markov Chain Monte Carlo (MCMC) sampling steps outlined by [3, 4, 27].

In this section, we have satisfied the privacy incentive. In Sec. 3, we assign a value v C to each coalition C ⊆ N 's perturbed SS o C ≜ { o i } i ∈ C that would decrease, on average, as the DP guarantee strengthens. In Secs. 4 and 5, we outline our reward scheme: Each party i will be rewarded with model parameters sampled from q i ( θ ) (in short, model reward ) for future predictions with an appropriate reward value r i (decided based on ( v C ) C ⊆ N ) to satisfy collaborative ML incentives (e.g., individual

rationality, fairness). Our work's main contributions, notations, and setup are detailed in Fig. 1. The main steps involved are detailed in Algo. 2.

Noise-awareinference,Bayesian surprise,KLestimation to computevc

Wantbettermodel

Wantstrongerprivacy

reward(higherr;)

outofovercaution

Perturbed SS, 0;

Every other

small e

Ei

Data quantity, C;

partyk's

DP noise distribution, p(Z)

(0k Ck p(Zk))

Rewarded model parameters drawn from q;(0)

Achievesafair,rationaltargetrewardvaluer*

Ensures (,≤ ek)-DPfor any other party k

Issimilarto thegrand coalition'sposterior(SeeFig.7)

p-Shapleyfairness,DPpost-processing,scaleSStotemper datalikelihood

3 Valuation of Perturbed Sufficient Statistics

The perturbed SS o C of coalition C is more valuable and assigned a higher value v C if it yields a model (in our work here, the DP noise-aware posterior p ( θ | o C ) ) of higher quality. Most data valuation methods [24, 18, 63] measure the quality of an ML model by its performance on a validation set. However, it may be challenging for collaborating parties (e.g., competing healthcare firms) to create and agree on a large, representative validation set as they may prioritize accurate predictions on different inputs (e.g., patient demographics) [47]. The challenge increases when each firm requires privacy and avoids data sharing. Other valuation methods [47, 59] have directly used the private inputs of the data (e.g., design matrix). Here, we propose to value the perturbed SS o C of coalition C based on the surprise [23] that it elicits from the prior belief of model parameters, as defined below:

Definition 3.1 (Valuation via Bayesian Surprise) . The value of coalition C or its surprise v C is the KL divergence D KL ( p ( θ | o C ); p ( θ )) between posterior p ( θ | o C ) vs. prior p ( θ ) .

From Def. 3.1, a greater surprise would mean that more bits will be needed to encode the information in p ( θ | o C ) given that others already know p ( θ ) . Otherwise, a smaller surprise means our prior belief has not been updated significantly. Moreover, as the valuation depends on the observed o C , the surprise elicited by the exact SS and data points will indirectly influence the valuation. Next, by exploiting the equality of the expected Bayesian surprise and the information gain on model parameters θ given perturbed SS o C (i.e., E o C [ v C ] = I ( θ ; o C ) ), we can establish the following essential properties of our valuation function:

V1 Non-negativity. ∀ C ⊆ N ∀ o C v C ≥ 0 . This is due to the non-negativity of KL divergence.

V2 Party monotonicity. In expectation w.r.t. o C , adding a party will not decrease the valuation: ∀ C ⊆ C ′ ⊆ N E o C ′ [ v C ′ ] ≥ E o C [ v C ] . The proof (App. C.1) uses the 'information never hurts' property.

V3 Privacy-valuation trade-off. When the DP guarantee is strengthened from ϵ i to a smaller ϵ s i and independent Gaussian noise is added to o i to generate o s i , in expectation, the value of any coalition C containing i will strictly decrease: Let v s C denote the value of coalition C with the random variable and realization of o i replaced by o s i . Then, ( i ∈ C ) ∧ ( ϵ s i < ϵ i ) ⇒ E o C [ v C ] > E o s C [ v s C ] .

The proof of V3 (App. C.1) uses the data processing inequality of information gain and the conditional independence between θ and o s i given o i . Together, these properties address an important question of how to ensure DP and value a party's data to enforce a privacy-valuation trade-off (Sec. 1). Additionally, in App. C.2, we prove that in expectation, our Bayesian surprise valuation is equivalent to the alternative valuation that measures the similarity of p ( θ | o C ) to the grand coalition N 's DP noise-aware posterior p ( θ | o N ) .

Implementation. Computing the Bayesian surprise valuation is intractable since the DP noise-aware posterior p ( θ | o C ) and its KL divergence from p ( θ ) do not have a closed-form expression. Nonetheless, there exist approximate inference methods like the Markov chain Monte Carlo (MCMC) sampling to estimate p ( θ | o C ) efficiently, as discussed in App. A.3. As our valuation function requires estimating the value of multiple coalitions and the posterior sampling step is costly, we prefer estimators with a low time complexity and a reasonable accuracy for a moderate number m of samples. We recommend KL estimation to be performed using the nearest-neighbors method [45], and repeated and averaged to reduce the variance of the estimate (see App. C.3 for a discussion). The nearest-neighbor KL estimator is also asymptotically unbiased; drawing more samples would reduce the bias and variance of our estimates and is more likely to ensure fairness - for example, party i 's sampled valuation is only larger than j 's if i 's true valuation is higher.

Remark. Our valuation is based on the submitted information { c i , o i , p ( Z i ) } i ∈ N without verifying or incentivizing their truthfulness. We discuss how this limitation is shared by existing works and can be overcome by legal contracts and trusted data-sharing platforms in App. I.

4 Reward Scheme for Ensuring Incentives

After valuation, the mediator should reward each party i with a model reward (i.e., consisting of samples from q i ( θ ) ) for future predictions. Concretely, q i ( θ ) is a belief of model parameters θ after learning from the perturbed SS o N . As in Sec. 3, we value party i 's model reward as the KL divergence from the prior: r i ≜ D KL ( q i ( θ ); p ( θ )) . The mediator will first decide the target reward value r ∗ i for every party i ∈ N using { v C } C ⊆ N to satisfy incentives such as fairness. The mediator will then control and generate a different q i ( θ ) for every party i ∈ N such that r i = r ∗ i using reward control mechanisms from Sec. 5. We will now outline the incentives and desiderata for model reward q i ( θ ) and reward values r i and r ∗ i for every party i ∈ N when the grand coalition forms 7 .

P1 DP-Feasibility. In party i 's reward, any other party k is still guaranteed at least its original ( λ, ϵ k ) -DP guarantee or stronger. The implication is that the generation of party i 's reward should not require more private information (e.g., SS) from party k .

P2 Efficiency. There is a party i ∈ N whose model reward is the grand coalition N 's posterior, i.e., q i ( θ ) = p ( θ | o N ) . It follows that r i = v N .

P3 Fairness. The target reward values ( r ∗ i ) i ∈ N must consider the coalition values { v C } C ⊆ N and satisfy properties F1 to F4 given in [47] and reproduced in App. D.2. The monotonicity axiom F4 ensures using a valuation function which enforces that a privacy-valuation trade-off will translate to a privacy-reward trade-off and deter parties from selecting excessive DP guarantees.

P4 Individual Rationality. Each party should receive a model reward that is more valuable than the model trained on its perturbed SS alone: ∀ i ∈ N r ∗ i ≥ v i .

P5 Similarity to Grand Coalition's Model. Among multiple model rewards q i ( θ ) whose value r i equates the target reward r ∗ i , we secondarily prefer one with a higher similarity r ′ i = -D KL ( p ( θ | o N ); q i ( θ )) to p ( θ | o N ) . 8

P6 Group Welfare. The reward scheme should maximize the total reward value ∑ n i =1 r i to increase the utility of model reward for each party and achieve the aims of collaborative ML.

Choice of desiderata. We adopt the desiderata from [47] but make P1 and P2 more specific (by considering each party's actual reward q i ( θ ) over just its values r i and v N ) and introduce P5. Firstly, for our Bayesian surprise valuation function, the feasibility constraint of [47] is inappropriate as removing a party or adding some noise realization may result in r i > v N , 9 so we propose P1 instead. Next, we recognize that party i is not indifferent to all model rewards q i ( θ ) with the same target reward value as they may have different utility (predictive performance). Thus, we propose our more specific P2 and a secondary desideratum P5. As P5 is considered after other desiderata, it does not conflict with existing results, e.g., design for ( r ∗ i ) i ∈ N to satisfy other incentives.

Remark on Rationality. In P4, a party's model reward is compared to the model trained its perturbed SS instead of its exact SS alone. This is because the mediator cannot access (and value the model

trained on) the private exact SS. Moreover, with no restrictions on the maximum DP noise, the value of some party's exact SS may exceed the grand coalition's perturbed SS when parties require strong DP guarantees. P4 is sufficient when parties require DP when alone to protect data from curious users of their ML model [1, 3, 27]. For example, a hospital may not want doctors to infer specific patients' data. When parties do not require DP when alone, our reward scheme cannot theoretically ensure that the model reward from collaboration is better than using the exact SS. We further discuss this limitation in App. D.3.

Design of ( r ∗ i ) i ∈ N . To satisfy the desiderata from [47] (including our fairness P3 and rationality P4 incentives), we adopt their ρ -Shapley fair reward scheme with ρ ∈ (0 , 1] that sets r ∗ i = v N ( ϕ i / max k ∈ N ϕ k ) ρ with Shapley value 10 ϕ i ≜ (1 /n ) ∑ C ⊆ N \ i [ ( n -1 | C | ) -1 ( v C ∪{ i } -v C ) ] . Shapley value's consideration of marginal contribution (MC) to all coalitions is key to ensuring strict desirability F3 such that party i obtains a higher reward than party k (despite v i = v k ) if i 's perturbed SS adds more value to every other non-empty coalition. Applying Theorem 1 of [47], the mediator should set ρ between 0 and min i ∈ N log( v i /v N ) / log( ϕ i / max k ϕ k ) to guarantee rationality. Selecting a larger ρ incentivizes a party with a high-quality perturbed SS to share by fairly limiting the benefits to parties with lower-quality ones. Selecting a smaller ρ reward parties more equally and increase group welfare P6. Refer to Sec. 4.2 of [47] for a deeper analysis of the impact of varying ρ . These results hold for any choice of ( λ, ϵ i ) .

After explaining the desiderata for model reward q i ( θ ) and reward values r i and r ∗ i for every party i ∈ N , we are now ready to solve for q i ( θ ) such that r i = r ∗ i .

5 Reward Control Mechanisms

This section discusses two mechanisms to generate model reward q i ( θ ) with different attained reward value r i for every party i ∈ N by controlling a single continuous parameter and solving for its value such that the attained reward value equates the target reward value: r i = r ∗ i . We will discuss the more obvious reward mechanism in Sec. 5.1 to contrast its cons with the pros of that in Sec. 5.2. Both reward mechanisms do not request new information from the parties; thus, the DP post-processing property applies, and every party k is still guaranteed at least its original DP guarantee or stronger in all model rewards (i.e., P1 holds).

5.1 Reward Control via Noise Addition

The work of [47] controls the reward values by adding Gaussian noise to the data outputs. We adapt it such that the mediator controls the reward value for party i ∈ N by adding Gaussian noise to the perturbed SS of each party k ∈ N instead. To generate the model reward for party i (superscripted), the mediator will reparameterize the sampled Gaussian noise vectors { e i k ∼ N ( 0 , I ) } k ∈ N to generate the further perturbed SS 11

where ∆ 2 2 ( g k ) is the squared ℓ 2 -sensitivity of function g k that computes the exact SS s k from dataset D k (Sec. 2). Then, the mediator rewards party i with samples of model parameters θ from the new DP noise-aware posterior q i ( θ ) = p ( θ | t i N ) .

Here, the scalar τ i ≥ 0 controls the additional noise variance and can be optimized via root-finding to achieve r i = r ∗ i . The main advantage of this reward control mechanism is its interpretation of strengthening party k 's DP guarantee in all parties' model rewards (see P1). For example, it can be derived that if ϵ k = ∞ , then party k will now enjoy ( λ, 1 /τ i ) -DP guarantee in party i 's reward instead. If ϵ k < ∞ , then party k will now enjoy a stronger ( λ, ϵ k / (1 + τ i ϵ k )) -DP guarantee since ϵ k / (1 + τ i ϵ k ) < ϵ k .

However, this mechanism has some disadvantages. Firstly, for the same scaled additional noise variance τ i , using different noise realizations { e i k } k ∈ N will lead to model reward q i ( θ ) with varying

similarity r ′ i to the grand coalition N 's posterior. The mechanism cannot efficiently select the best model reward with higher r ′ i (P5). Secondly, the value of r i computed using such t i k may be nonmonotonic 12 in τ i (see Fig. 2d), which makes it hard to bracket the smallest root τ i that solves for r i = r ∗ i . To address these disadvantages, we will propose the next mechanism.

5.2 Reward Control via Likelihood Tempering

Intuitively, a party i who is assigned a lower target reward value r ∗ i < v N should be rewarded with posterior samples of model parameters θ that use less information from the datasets and SS of all parties. Sparked by the diffuse posterior algorithm [16], we propose that the mediator can generate such 'less informative' samples for party i using the normalized posterior 13

involving the product of the prior p ( θ ) and the data likelihood p ( D N | θ ) to the power of (or, said in another way, tempered by a factor of) κ i . Notice that setting κ i = 0 and κ i = 1 recover the prior p ( θ ) and the posterior p ( θ |D N ) , respectively. Thus, setting κ i ∈ (0 , 1) should smoothly interpolate between both. We can optimize κ i to control q i ( θ ) so that r i = r ∗ i < v N .

But, how do we temper the likelihood? We start by examining the easier, non-private setting. In Sec. 2, we stated that under our assumptions, the posterior p ( θ |D N ) can be computed by using the sum of data quantities { c k } k ∈ N and sum of exact SS s N . In App. E, we further show that using the tempered likelihood [ p ( D N | θ )] κ i is equivalent to scaling the data quantities and the exact SS s N by the factor κ i beforehand. In the private setting, the mediator can similarly scale the data quantities, the perturbed SS in o N (instead of the inaccessible exact SS), and the ℓ 2 -sensitivity by the factor κ i beforehand; see App. E.3 for details. This likelihood tempering mechanism addresses both disadvantages of Sec. 5.1:

There is no need to sample additional DP noise. We empirically show that tempering the likelihood produces a model reward that interpolates between the prior vs. posterior (in App. G) and preserves a higher similarity r ′ i to the grand coalition N 's posterior (P5 and hence, more group welfare P6) and better predictive performance than noise addition (see Sec. 6).

Using a smaller tempering factor κ i ∈ [0 , 1] provably decreases the attained reward value r i (see App. E). Thus, as the relationship between r i and κ i is monotonic, we can find the only root by searching the interval [0 , 1] .

Remark. Our discussion on improving the estimate of v C in the paragraph on implementation in Sec. 3 also applies to the estimate of r i in Secs. 5.1 and 5.2. Thus, solving for τ i or κ i to achieve r i = r ∗ i using any root-finding algorithm can only be accurate up to the variance in our estimate.

6 Experiments and Discussion

This section empirically evaluates the privacy-valuation and privacy-reward trade-offs (Sec. 6.1), reward control mechanisms (Sec. 6.2), and their relationship with the utility of the model rewards (Sec. 6.3). The time complexity of our scheme is analyzed in App. F and baseline methods are discussed in App. H.3. We consider Bayesian linear regression (BLR) with unknown variance on the Syn and CaliH datasets, and Bayesian logistic regression on the Diab dataset with 3 collaborating parties (see App. H.1 for details) and enforce (2 , ϵ i ) -Rényi DP. For Synthetic BLR (Syn) , we select and use a normal inverse-gamma distribution (i) to generate the true regression model weights, variance, and a 2 D-dataset and (ii) as our model prior p ( θ ) . We consider 3 parties with c 1 = 100 , c 2 = 200 , c 3 = 400 data points, respectively. For Californian Housing dataset (CaliH) [44], as in [47], 60% of the CaliH data is deemed 'public/historic' and used to pre-train a neural network without DP. Real estate firms may only care about the privacy of their newest transactions. As the parties' features-house values relationship may differ from the 'public' dataset, we do transfer learning and selectively retrain only the last layer with BLR using the parties' data. Parties 1 to 3 have,

-

2

-

1

0

1

log10

/epsilon1

2

0

2

4

6

8

v

2

φ

2

r

2

vN

-

2

-

1

0

1

log10

/epsilon1

2

0

4

8

12

16

v

2

φ

2

r

2

vN

-

2

-

1

0

1

log10

/epsilon1

2

0

2

4

v

2

φ

2

r

2

vN

0

7

14

log2

τi

2

3

4

5

ri

0

.

0

0

.

5

1

.

0

κi

0

5

10

log2

τi

2

3

4

5

ri

0

.

0

0

.

5

1

.

0

κi

2

4

ri

-

6

-

4

-

2

0

r

′

i

κi

τi

(a) Syn

(b) CaliH

(c) Diab

(d)

r

i

vs.

κ

i

,

τ

i

(Non-monotonic in

τ

i

)

(e)

r

i

vs.

κ

i

,

τ

i

(Monotonic in

τ

i

)

(f)

r

′

i

vs.

r

i

respectively, 20% , 30% , and 50% of the dataset with 6581 data points and 6 features. For PIMA Indian Diabetes classification dataset (Diab) [50], we use a Bayesian logistic regression model to predict whether a patient has diabetes based on sensitive inputs (e.g., patient's age, BMI, number of pregnancies). To reduce the training time, we only use the 4 PCA main components as features (to generate the approximate SS) [27]. Parties 1 , 2 , and 3 have, respectively, 20% , 30% , and 50% of the dataset with 614 data points. As we are mainly interested 14 in the impact of one party controlling its privacy guarantee ϵ i , for all experiments, we only vary party 2 's from the default 0 . 1 . We fix the privacy guarantees of others ( ϵ 1 = ϵ 3 = 0 . 2 ) and ρ = 0 . 2 in the ρ -Shapley fair reward scheme, and analyze party 2 's reward and utility. Note that as ϵ 2 increases (decreases), party 2 becomes the most (least) valuable of all parties.

6.1 Privacy-valuation and Privacy-reward Trade-offs

For each dataset, we only vary the privacy guarantee of party i = 2 with ϵ 2 ∈ [0 . 004 , 0 . 02 , 0 . 1 , 0 . 5 , 2 . 5 , 12 . 5] and use the Gaussian mechanism and a fixed random seed to generate the perturbed SS o 2 from the exact SS s 2 . Fig. 2a-c plot the mean and shades the standard error of v i , v N , ϕ i , and r i over 5 runs. The privacy-valuation and privacy-reward trade-offs can be observed: As the privacy guarantee weakens (i.e., ϵ 2 increases), party 2 's valuation v 2 , Shapley value ϕ 2 , and attained reward value r 2 increase. When ϵ 2 is large, party 2 will be the most valuable contributor and rewarded with p ( θ | o N ) , hence attaining r i = v N . App. H.5 shows that the trade-offs do not hold for non-noise-aware inference.

6.2 Reward Control Mechanisms

We use the Syn experiment to compare the reward mechanisms that vary the noise addition using τ i (Sec. 5.1) vs. temper the likelihood using κ i (Sec. 5.2). The mechanisms control q i ( θ ) (i.e., used to generate party i 's model reward) to attain the target reward values. For each value of τ i and κ i considered, we repeat the posterior sampling and KL estimation method 5 times. Figs. 2d and 2e-f use different sets of sampled noise { e i k } k ∈ N to demonstrate the stochastic relationship between r i and τ i . In Fig. 2d, the non-monotonic disadvantage of noise addition can be observed: As τ i increases, r i does not consistently decrease, hence making it hard to solve for the smaller τ i that attains r ∗ i = 3 . In contrast, as κ i decreases from 1 , r i consistently decreases. Furthermore, in Fig. 2f, we demonstrate the other advantage of likelihood tempering: For the same r i , tempering the likelihood leads to a higher similarity r ′ i to the posterior p ( θ | o N ) than noise addition. In App. H.6, we report the relationship between r i vs. κ i and τ i for the other real-world datasets.

6.3 Utility of Model Reward

The utility (or the predictive performance) of both Bayesian models can be assessed by the mean negative log probability (MNLP) of a non-private test set. 15 In short, MNLP reflects how unlikely the test set is given the perturbed SS and additionally cares about the uncertainty/confidence in the model predictions (e.g., due to the impact of DP noise). MNLP will be higher (i.e., worse) when the model

-

2

0

log10

/epsilon1

2

0

.

0

0

.

5

MNLP

N

r

i

=2

-

2

0

log10

/epsilon1

2

0

.

8

1

.

2

1

.

6

MNLP

N

r

i

=2

-

2

0

log10

/epsilon1

2

0

.

60

0

.

65

0

.

70

MNLP

N

r

i

=2

2

4

ri

0

.

0

0

.

5

MNLPr

κi

τi

5

10

ri

1

.

0

1

.

5

MNLPr

κi

τi

1

2

ri

0

.

6

0

.

7

MNLPr

τi

κi

(a) Syn (vs.

ϵ

2

)

(b) CaliH (vs.

ϵ

2

)

ϵ

2

)

(d) Syn (vs.

r

i

)

(e) CaliH (vs.

r

i

)

r

i

)

is more uncertain of its accurate predictions or overconfident in inaccurate predictions on the test set; see App. H.2 for an in-depth definition.

Privacy trade-offs. Figs. 3a-c illustrate the privacy-utility trade-off described in Sec. 1: As ϵ 2 decreases (i.e., privacy guarantee strengthens), the MNLP N of grand coalition N 's collaboratively trained model and the MNLP i of party i = 2 's individually trained model generally increase, so their utilities drop (†). This motivates the need to incentivize party 2 against selecting an excessively small ϵ 2 by enforcing privacy-valuation and privacy-reward trade-offs. From Figs. 3a-c, the impact of our scheme enforcing the trade-offs can be observed: As ϵ 2 decreases, the MNLP r of party i = 2 's model reward increases.

Remark. In Fig. 3c, an exception to (†) is observed. The exception illustrates that the privacy-valuation trade-off may not hold for a valuation function based on the performance on a validation set.

Individual rationality. It can be observed from Figs. 3a-c that as ϵ 2 decreases, the MNLP r of party i = 2 's model reward increases much less rapidly than the MNLP i of its individually trained model. So, it is rational for party i = 2 to join the collaboration to get a higher utility.

Remark. Party i = 2 's utility gain appears small when ϵ 2 is large due to parties 1 and 3 's selection of strong privacy guarantee ϵ = 0 . 2 . Party i can gain more when other parties require weaker privacy guarantees such as ϵ = 2 instead (see App. H.5).

Likelihood tempering is a better reward control mechanism. Extending Sec. 6.2, we compare the utility of party i 's model reward generated by noise addition vs. likelihood tempering in Figs. 3d-f. Across all experiments, likelihood tempering (with κ i ) gives (i) a lower MNLP r and hence a higher utility, and (ii) a lower variance in MNLP r than varying the noise addition (with τ i ).

7 Related Works

Fig. 5 in App. B gives a diagrammatic overview showing how our work fills the gap in existing works.

Data Valuation. Most data valuation methods are not differentially private and directly access the data. For example, computing the information gain [47] or volume [59] requires the design matrix. While it is possible to make these valuation methods differentially private (see App. H.3) or value DP trained models using validation accuracy (on an agreed, public validation set), the essential properties of our valuation function (V2-V3) may not hold.

Privacy Incentive. Though the works of [20, 60] reward parties directly proportional to their privacy budget, their methods do not incentivize data sharing as a party does not fairly receive a higher reward value for contributing a larger, more informative dataset. While the work of [28] artificially creates a privacy-reward trade-off by paying each party i the product of its raw data's Shapley value ϕ i and a monotonic transformation of ϵ i , it neither ensures DP w.r.t. the mediator nor fairly considers how a stronger DP guarantee may reduce party i 's marginal contribution to others (hence ϕ i ). The work of [13] considers data acquisition from parties with varying privacy requirements but focuses on the mean estimation problem and designing payment and privacy loss functions to get parties to report their true unit cost of privacy loss. Our work here distinctively considers Bayesian models and fairness and allows parties to choose their privacy guarantees directly while explicitly enforcing a privacy-reward trade-off.

Difficulties ensuring incentives with existing DP/FL works. The one posterior sampling (OPS) method [56, 16] proposes that each party i can achieve DP by directly releasing samples from the posterior p ( θ |D i ) (if the log-likelihood is bounded). However, OPS is data inefficient and may not guarantee privacy for approximate inference [15]. It is unclear how we can privately value a coalition C and sample from the joint posterior p ( θ |{D i } i ∈ C ) . DP-FedAvg/DP-FedSGD [36] or DP-SGD [1] enable collaborative but private training of neural networks by requiring each party i to clip and add Gaussian noise to its submitted gradient updates. However (in addition to the valuation function issue above), it is tricky to ensure that the parties' rewards satisfy data sharing incentives. In each round of FL, parties selected will receive the (same) latest model parameters to compute gradient updates. This setup goes against the fairness (P3) incentive as parties who share less informative gradients should be rewarded with lower quality model parameters instead. Although the unfairness may potentially be corrected via gradient-based [58] or monetary rewards, there is no DP reward scheme to guarantee a party better model reward from collaboration than in solitude or a higher monetary reward than its cost of participation, hence violating individual rationality.

8 Conclusion

Unlike existing works in collaborative ML that solely focus on the fairness incentive, our proposed scheme further (i) ensures privacy for the parties during valuation and in model rewards and (ii) enforces a privacy-valuation trade-off to deter parties from unfetteredly selecting excessive DP guarantees to maximize the utility of collaboratively trained models. 16 This involves novelly combining our proposed Bayesian surprise valuation function and reward control mechanism with DP noise-aware inference. We empirically evaluate our scheme on several datasets. Our likelihood tempering reward control mechanism consistently preserves better predictive performance.

Our work has two limitations which future work should overcome. Firstly, we only consider ML models with SS (see App. A.1 for applications) and a single round of sharing information with the mediator as a case study to show the incentives and trade-offs can be achieved. Future work should generalize our scheme to ML models without an explicit SS.

Next, like the works of [18, 17, 25, 40, 47, 51] and others, we do not consider the truthfulness of submitted information and value data as-is . This limitation is acceptable for two reasons. 1) Parties such as hospitals and firms will truthfully share information as they are primarily interested in building and receiving a model reward of high quality and may additionally be bound by the collaboration's legal contracts and trusted data-sharing platforms. For example, with the use of X-road ecosystem, 17 parties can upload a private database which the mediator can query for the perturbed SS. This ensures the authenticity of the data (also used by the owner) and truthful computation given the uploaded private database. 2) Each party would be more inclined to submit true information as any party k who submits fake SS will reduce its utility from the collaboration. This is because party k 's submitted SS is used to generate k 's model reward and cannot be replaced locally as party k will only receive posterior samples. Future work should seek to verify and incentivize truthfulness.

Acknowledgments and Disclosure of Funding

This research/project is supported by the National Research Foundation Singapore and DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG 2 -RP2020 -018 ). This work is also supported by the National Natural Science Foundation of China (62206139) and the Major Key Project of PCL (PCL2023AS7-1).

References

M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proc. ACM CCS , pages 308-318, 2016.

B. Balle and Y.-X. Wang. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In Proc. ICML , pages 394-403, 2018.

G. Bernstein and D. Sheldon. Differentially private Bayesian inference for exponential families. In Proc. NeurIPS , pages 2924-2934, 2018.

G. Bernstein and D. Sheldon. Differentially private Bayesian linear regression. In Proc. NeurIPS , pages 525-535, 2019.

C. M. Bishop. Pattern Recognition and Machine Learning . Springer, 2006.

Center for Open Data Enterprise. Sharing and utilizing health data for AI applications. Roundtable report, 2019.

Y. Chen, Y. Shen, and S. Zheng. Truthful data acquisition via peer prediction. In Proc. NeurIPS , 2020.

K. Choi, M. Liao, and S. Ermon. Featurized density ratio estimation. In Proc. UAI , pages 172-182, 2021.

J. Conway. Artificial Intelligence and Machine Learning: Current Applications in Real Estate . PhD thesis, Massachusetts Institute of Technology, 2018.

T. Cover and J. Thomas. Elements of Information Theory . John Wiley & Sons, Inc., 1991.

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Proc. TCC , pages 265-284, 2006.

C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407, 2014.

A. Fallah, A. Makhdoumi, A. Malekian, and A. Ozdaglar. Optimal and differentially private data acquisition: Central and local mechanisms. arXiv:2201.03968, 2022.

T. Farrand, F. Mireshghallah, S. Singh, and A. Trask. Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy. In Proc. ACM CCS Workshop on Privacy-Preserving Machine Learning in Practice , pages 15-19, 2020.

J. Foulds, J. Geumlek, M. Welling, and K. Chaudhuri. On the theory and practice of privacypreserving Bayesian data analysis. In Proc. UAI , pages 192-201, 2016.

J. Geumlek, S. Song, and K. Chaudhuri. Rényi differential privacy mechanisms for posterior sampling. In Proc. NIPS , pages 5289-5298, 2017.

A. Ghorbani, M. P. Kim, and J. Zou. A distributional framework for data valuation. In Proc. ICML , pages 3535-3544, 2020.

A. Ghorbani and J. Zou. Data Shapley: Equitable valuation of data for machine learning. In Proc. ICML , pages 2242-2251, 2019.

M. D. Hoffman and A. Gelman. The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. JMLR , 15(1):1593-1623, 2014.

R. Hu and Y. Gong. Trading data for learning: Incentive mechanism for on-device federated learning. In Proc. GLOBECOM , pages 1-6, 2020.

J. Huggins, R. P. Adams, and T. Broderick. PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference. In Proc. NIPS , pages 611--3621, 2017.

N. Hynes, D. Dao, D. Yan, R. Cheng, and D. Song. A demonstration of Sterling: A privacypreserving data marketplace. Proc. VLDB Endowment , 11(12):2086-2089, 2018.

L. Itti and P. Baldi. Bayesian surprise attracts human attention. Vision Research , 49(10):12951306, 2009.

R. Jia, D. Dao, B. Wang, F. A. Hubis, N. M. Gurel, B. Li, C. Zhang, C. Spanos, and D. Song. Efficient task-specific data valuation for nearest neighbor algorithms. Proc. VLDB Endowment , 12(11):1610-1623, 2019.

R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. Gurel, B. Li, C. Zhang, D. Song, and C. Spanos. Towards efficient data valuation based on the Shapley value. In Proc. AISTATS , pages 1167-1176, 2019.

T. Kulkarni, J. Jälkö, S. Kaski, and A. Honkela. Locally differentially private Bayesian inference. arXiv:2110.14426, 2021.

T. Kulkarni, J. Jälkö, A. Koskela, S. Kaski, and A. Honkela. Differentially private Bayesian inference for generalized linear models. In Proc. ICML , pages 5838-5849, 2021.

J. Liu, J. Lou, J. Liu, L. Xiong, J. Pei, and J. Sun. Dealer: an end-to-end model marketplace with differential privacy. Proc. VLDB Endowment , 14(6):957-969, 2021.

Y. Liu and J. Wei. Incentives for federated learning: A hypothesis elicitation approach. arXiv:2007.10596, 2020.

O. Lobel. The problem with too much data privacy. Time Magazine , 27 Oct 2022.

H. Lv, Z. Zheng, T. Luo, F. Wu, S. Tang, L. Hua, R. Jia, and C. Lv. Data-free evaluation of user contributions in federated learning. In Proc. IEEE WiOpt , 2021.

L. Lyu, Y. Li, K. Nandakumar, J. Yu, and X. Ma. How to democratise and protect AI: Fair and differentially private decentralised deep learning. IEEE Transactions on Dependable and Secure Computing , 19(2):1003-1017, 2022.

M. Maschler and B. Peleg. A characterization, existence proof and dimension bounds for the kernel of a game. Pacific J. Mathematics , 18(2):289-328, 1966.

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-efficient learning of deep networks from decentralized data. In Proc. AISTATS , pages 1273-1282, 2017.

B. McMahan and A. Thakurta. Federated learning with formal differential privacy guarantees. Google AI Blog, Google Research, Feb 2022.

H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang. Learning differentially private recurrent language models. In Proc. ICLR , 2018.

J. W. Miller and D. B. Dunson. Robust Bayesian inference via coarsening. Journal of the American Statistical Association , 2018.

I. Mironov. Rényi differential privacy. In Proc. 30th IEEE Computer Security Foundations Symposium (CSF) , pages 263-275, 2017.

K. P. Murphy. Probabilistic Machine Learning: An Introduction . MIT Press, 2022.

Q. P. Nguyen, B. K. H. Low, and P. Jaillet. Trade-off between payoff and model rewards in Shapley-fair collaborative machine learning. In Proc. NeurIPS , 2022.

F. Nielsen and R. Nock. Entropies and cross-entropies of exponential families. In Proc. IEEE ICIP , pages 3621-3624, 2010.

Y.-K. Noh, M. Sugiyama, S. Liu, M. C. Plessis, F. C. Park, and D. D. Lee. Bias reduction and metric learning for nearest-neighbor estimation of Kullback-Leibler divergence. In Proc. AISTATS , pages 669-677, 2014.

Ocean Protocol Foundation. Ocean Protocol: A decentralized substrate for AI data and services. Technical whitepaper, 2019.

R. K. Pace and R. Barry. Sparse spatial auto-regressions. Statistics and Probability Letters , 33(3):291-297, 1997.

F. Pérez-Cruz. Kullback-Leibler divergence estimation of continuous distributions. In Proc. IEEE ISIT , pages 1666-1670, 2008.

R. H. L. Sim, X. Xu, and B. K. H. Low. Data valuation in machine learning: 'ingredients', strategies, and open challenges. In Proc. IJCAI , pages 5607-5614, 2022.

R. H. L. Sim, Y. Zhang, M. C. Chan, and B. K. H. Low. Collaborative machine learning with incentive-aware model rewards. In Proc. ICML , pages 8927-8936, 2020.

A. Singh and D. Koutra. Lecture 16 of 10-704: Information processing and learning course. Carnegie Mellon University, 19 March 2015.

S. Singh and B. Póczos. Finite-sample analysis of fixedk nearest neighbor density functional estimators. In Proc. NIPS , 2016.

J. W. Smith, J. E. Everhart, W. C. Dickson, W. C. Knowler, and R. S. Johannes. Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proc. Annu. Symp. Comput. Appl. Med. Care , pages 261-265, 1988.

S. S. Tay, X. Xu, C. S. Foo, and B. K. H. Low. Incentivizing collaboration in machine learning via synthetic data rewards. In Proc. AAAI , pages 9448-9456, 2022.

M. Titsias. Variational learning of inducing variables in sparse Gaussian processes. In Proc. AISTATS , pages 567-574, 2009.

J. T. Wang and R. Jia. A note on "Towards efficient data valuation based on the shapley value'. arXiv:2302.11431, 2023.

Q. Wang, S. R. Kulkarni, and S. Verdú. Divergence estimation for multidimensional densities via k -nearest-neighbor distances. IEEE Transactions on Information Theory , 55(5):2392-2405, 2009.

T. Wang, J. Rausch, C. Zhang, R. Jia, and D. Song. A principled approach to data valuation for federated learning. In Q. Yang, L. Fan, and H. Yu, editors, Federated Learning , volume 12500 of Lecture Notes in Computer Science , pages 153-167. Springer, Cham, 2020.

Y. X. Wang, S. Fienberg, and A. Smola. Privacy for free: Posterior sampling and stochastic gradient Monte Carlo. In Proc. ICML , pages 2493-2502, 2015.

L. Watson, R. Andreeva, H. Yang, and R. Sarkar. Differentially private Shapley values for data evaluation. arXiv:2206.00511, 2022.

X. Xu, L. Lyu, X. Ma, C. Miao, C. S. Foo, and B. K. H. Low. Gradient driven rewards to guarantee fairness in collaborative machine learning. In Proc. NeurIPS , pages 16104-16117, 2021.

X. Xu, Z. Wu, C. S. Foo, and B. K. H. Low. Validation free and replication robust volume-based data valuation. In Proc. NeurIPS , pages 10837-10848, 2021.

Y. Xu, M. Xiao, H. Tan, A. Liu, G. Gao, and Z. Yan. Incentive mechanism for differentially private federated learning in Industrial Internet of Things. IEEE Transactions On Industrial Informatics , 18(10):6927-6939, 2022.

M. Yang, L. Lyu, J. Zhao, T. Zhu, and K.-Y. Lam. Local differential privacy and its applications: A comprehensive survey. arXiv:2008.03686, 2020.

Q. Yang, Y. Liu, T. Chen, and Y. Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology , 10(2):1-19, 2019.

J. Yoon, S. O. Arik, and T. Pfister. Data valuation using reinforcement learning. In Proc. ICML , pages 10842-10851, 2020.

L. Zhang, T. Zhu, P. Xiong, W. Zhou, and P. S. Yu. More than privacy: Adopting differential privacy in game-theoretic mechanism design. ACM Comput. Surv. , 54(7), 2022.

B. Z. H. Zhao, M. A. Kaafar, and N. Kourtellis. Not one but many tradeoffs: Privacy vs. utility in differentially private machine learning. In Proc. ACM SIGSAC Conference on Cloud Computing Security Workshop , pages 15-26, 2020.

P. Zhao and L. Lai. Minimax optimal estimation of KL divergence for continuous distributions. IEEE Transactions on Information Theory , 66(12):7787-7811, 2020.

A Fundamental Concepts

In this section, we will elaborate on concepts from Sec. 2 in more detail.

A.1 Sufficient Statistics

Bayesian learning involves updating our belief of the likely values of the model parameters θ , captured in the prior p ( θ ) , to a posterior belief p ( θ |D i ) ∝ p ( θ ) × p ( D i | θ ) . The posterior belief gets more concentrated (around the maximum likelihood estimate) after observing a larger dataset D i .

The statistic s i is a SS for D i if θ and D i are conditionally independent given s i , i.e., p ( θ |D i ) = p ( θ | s i , D i ) = p ( θ | s i ) [48, 52]. Knowing the dataset D i does not provide any extra information about θ beyond the SS s i . SS exists for exponential family models [5] and Bayesian linear regression [39]. Approximate SS has been proposed by [21] for generalized linear models. For more complex data such as images, we can use pre-trained neural networks like VGG16 as feature extractors and generate SS from the last hidden layer's outputs.

Bayesian Linear Regression. In linear regression, each datum consists of the input x ∈ R w and the output variable y ∈ R . Let D denote the dataset with c data points, and y and X be the corresponding concatenated output vector and design matrix in R c × w . Bayesian linear regression models the relationship as y = Xw + N (0 , σ 2 I ) where the model parameters θ consists of the weight parameters w ∈ R w and the noise variance σ 2 . The likelihood

only depends on data via the sufficient statistics s = ( y ⊤ y , X ⊤ y , X ⊤ X ) . Concretely, when the prior p ( θ ) of the weights and variance follow a normal inverse-gamma distribution, NIG ( 0 , V 0 , a 0 , b 0 ) , the posterior p ( θ |D i ) is the normal inverse-gamma distribution NIG ( w i , V i , a 0 + c i / 2 , b i ) where c i is the number of data points and

can be computed directly from s i . The posterior belief p ( θ |D i , D j ) given parties i and j 's dataset can be similarly computed using the SS of their pooled dataset, s ij . As the SS s ij works out to s i + s j , we only need s i and s j from party i and j instead of their private datasets.

Generalized Linear Model (GLM). A generalized linear model (GLM) generalizes a linear model by introducing an inverse link function Υ . The probability of observing the output y given input x = ( x (1) , . . . , x ( w ) ) and model weights θ depends on their dot product

Next, we define the GLM mapping function υ to the log-likelihood of observing y given the GLM model. Formally,

As an example, logistic regression is a GLM with Υ defined as the sigmoid function and p ( y = ± 1 | sigmoid ( x ⊤ θ )) follows a Bernoulli distribution. As the non-linearity of Υ disrupts the exponential family structure, logistic regression and other GLMs do not have sufficient statistics. Logistic regression's GLM mapping function υ log ( y, x ⊤ θ ) = -log(1 + exp( -y x ⊤ θ )) .

[21] propose to approximate the GLM mapping function υ with an M -degree polynomial approximation υ M . υ M is an exponential family model with sufficient statistics g ( d ) = {∏ w i =1 ( yx ( i ) ) m i | ∑ i m i ≤ M, ∀ i m i ∈ Z + 0 } . These SS are the polynomial approximate sufficient statistics for GLMs. For example, when M = 2 and x = ( x (1) , x (2) ) , g ( d ) = [ 1 , x (1) y, x (2) y, x 2 (1) y 2 , x 2 (2) y 2 , x (1) x (2) y 2 ] .

A.2 Differential Privacy

Remark 1. Our work aims to ensure example-level DP for each collaborating party: A party updating/adding/deleting a single datum will only change the perturbed SS visible to the mediator and the corresponding belief of the model parameters in a provably minimal way. We are not ensuring user-level DP : The belief of model parameters only changes minimally after removing a collaborating party's (or a user/data owner's) dataset, possibly with multiple data points [35].

Intuitively, a DP algorithm R : D → o guarantees that each output o is almost equally likely regardless of the inclusion or exclusion of a data point d in D . This will allay privacy concerns and incentivize a data owner to contribute its data point d since even a knowledgeable attacker cannot infer the presence or absence of d .

The works on noise-aware inference [4, 27] assume that the input x and output y of any data point have known bounded ranges. We will start by introducing our domain-dependent definitions:

Definition A.1 (Neighboring datasets) . Two datasets D and D ′ are neighboring if D ′ can be obtained from D by replacing a single data point. The total number of data points and all other data points are the same.

Definition A.2 (Sensitivity [11]) . The sensitivity of a function g that takes in dataset D k quantifies the maximum impact a data point can have on the function output. The ℓ 1 -sensitivity ∆ 1 ( g ) and ℓ 2 -sensitivity ∆ 2 ( g ) measure the impact using the ℓ 1 and ℓ 2 norm, respectively. Given that D ′ i must be a neighboring dataset of D i ,

In our problem, g computes the exact SS s i for D i . The sensitivity can be known/computed if the dataset is normalized and the feature ranges are bounded.

We start with the definition of ϵ -differential privacy. The parameter ϵ bounds how much privacy is lost by releasing the algorithm's output.

Definition A.3 (Pure ϵ -DP [11]) . A randomized algorithm R : D → o with range O is ϵ -DP if for all neighboring datasets D and D ′ and possible output subset O ⊂ Range ( R ) ,

The Laplace mechanism [11] is an ϵ -DP algorithm. Instead of releasing the exact SS s i , the mechanism will output a sample of the perturbed SS o i ∼ Laplace( s i , (∆ 1 ( g ) /ϵ ) I ) .

A common relaxation of ϵ -differential privacy is ( ϵ, δ ) -differential privacy. It can be interpreted as ϵ -DP but with a failure of probability at most δ .

Definition A.4 ( ( ϵ, δ ) -DP) . A randomized algorithm R : D → o with range O is ( ϵ, δ ) -differentially private if for all neighboring datasets D and D ′ and possible output subset O ⊂ Range ( R ) ,

The Gaussian mechanism is an ( ϵ, δ ) -DP algorithm. The variance of the Gaussian noise to be added can be computed by the analytic Gaussian mechanism algorithm [2].

In the main paper, we have also discussed another relaxation of ϵ -differential privacy that is reproduced below:

Definition A.5 (Rényi DP [38]) . A randomized algorithm R : D → o is ( λ, ϵ ) -Rényi differentially private if for all neighboring datasets D and D ′ , the Rényi divergence of order λ > 1 is D λ ( R ( D ) || R ( D ′ )) ≤ ϵ where

When λ = ∞ , Rényi DP becomes pure ϵ -DP. Decreasing λ emphasizes less on unlikely large values and emphasizes more on the average value of the privacy loss random variable log [ P ( R ( D ) = o ) /P ( R ( D ′ ) = o )] with o ∼ R ( D ′ ) .

The Gaussian mechanism is a ( λ, ϵ ) -Rényi DP algorithm. Instead of releasing the exact SS s i , the mechanism will output a sample of the perturbed SS o i ∼ N ( s i , 0 . 5 ( λ/ϵ ) ∆ 2 2 ( g ) I ) .

Post-processing. A common and important property of all DP algorithms/mechanisms is their robustness to post-processing : Processing the output of a DP algorithm R without access to the underlying dataset will retain the same privacy loss and guarantees [12].

Choosing Rényi-DP over ( ϵ, δ ) -DP. In our work, we consistently use the Gaussian mechanism in all the experiments, like in that of [27]. We choose Rényi DP over ( ϵ, δ ) -DP due to the advantages stated below:

Rényi-DP is a stronger DP notion according to [38]: While ( ϵ, δ ) -DP allows for a complete failure of privacy guarantee with probability of at most δ , Rényi-DP does not and the privacy bound is only loosened more for less likely outcomes. Additionally, [38] claims that it is harder to analyze and optimize ( ϵ, δ ) -DP due to the trade-off between ϵ and δ . More details can be found in [38].

Rényi-DP supports easier composition: In a collaborative ML framework, each party i may need to release multiple outputs on the same dataset D i such as the SS and other information for preprocessing steps (e.g., principal component analysis). Composition rules bound the total privacy cost ˆ ϵ of releasing multiple outputs of differentially private mechanisms. It is harder to keep track of the total privacy cost when using ( ϵ, δ ) -DP due to advanced composition rules and the need to choose from a wide selection of possible ( ϵ ( δ ) , δ ) [38]. In contrast, the composition rule (i.e., Proposition 1 in [38]) is straightforward: When λ is a constant, the ϵ of different mechanisms can simply be summed.

Note that the contribution of our work will still hold for ( ϵ, δ ) -DP (using the Gaussian mechanism) and ϵ -DP (using the Laplace mechanism) with some modifications of the inference process and proofs.

Remark 2. Our work is in the same spirit as local DP (and we also think that no mediator can be trusted to directly access any party's private dataset) but does not strictly satisfy the definition of local DP (see Def. A.6). In the definition, the local DP algorithm takes in a single input/datum and ensures the privacy of its output - the perturbation mechanism is applied to every input independently. In contrast, in our case, a party may have multiple inputs and the perturbation mechanism is only applied to their aggregate statistics. Thus, a datum owner (e.g., a patient of a collaborating hospital) enjoys weaker privacy in our setting than the local DP setting.

Definition A.6 ( ϵ -Local DP [61]) . A randomized algorithm R is ϵ -local DP if for any pair of data points d, d ′ ∈ D and for any possible output O ⊂ Range ( R ) ,

A.3 DP Noise-Aware Inference

DP mechanisms introduce randomness and noise to protect the output of a function. Noise-naive techniques ignore the added noise in downstream analysis. In contrast, noise-aware techniques account for the noise added by the DP mechanism.

Consider a probabilistic model where the model parameters θ generate the dataset D i which then generates the exact and perturbed sufficient statistics of each party i , which are modeled as random variables S i and O i , respectively. The exact SS s i and perturbed SS o i computed by party i are realizations of S i and O i , respectively. As the mediator cannot observe i 's exact SS, S i is a latent random variable. Instead, the mediator observes i 's perturbed SS O i which also contains noise Z i added by the DP mechanism, i.e., O i ≜ S i + Z i . The Gaussian mechanism to ensure ( λ, ϵ ) -Rényi DP sets Z i ∼ N ( 0 , 0 . 5 ( λ/ϵ ) ∆ 2 2 ( g ) I ) . We depict the graphical model of our multi-party setting in Fig. 4.

Differences between exact, noise-naive and noise-aware inference. When the mediator observes the exact SS s i from party i , the exact posterior belief p ( θ | S i = s i ) can be computed in closed form based on App. A.1. However, when the mediator only observes the perturbed SS o i , the mediator can

θ

S

i

O

i

X

i

Z

i

ω

n

only compute the noise-naive and noise-aware posterior beliefs instead. The noise-naive posterior belief p ( θ | S i = o i ) will neither reflect the unobservability of the exact SS random variable S i accurately nor quantify the impact of the DP noise Z i . In contrast, for any party, the DP noise-aware posterior belief p ( θ | O i = o i ) , conveniently abbreviated as p ( θ | o i ) , will quantify the impact of the noise added by the DP mechanism. (For a coalition C of parties, the DP noise-aware posterior belief is p ( θ | o C ) ≜ p ( θ |{ O i = o i } i ∈ C )) , as described in Footnote 6.) The works of [3, 4, 27] have shown that DP noise-aware inference leads to a posterior belief that is better calibrated (i.e., lower bias and better quantification of uncertainty without overconfidence) and of higher utility (i.e., closer to the non-private posterior belief), thus a better predictive performance.

The main challenge of noise-aware inference lies in tractably approximating the integral p ( O i = o i | θ ) = ∫ p ( s i | θ ) p ( o i | s i ) d s i and p ( s i | θ ) = ∫ {D =( X , y ): g ( D )= s i } p ( X , y | θ ) d X d y over all datasets. [3, 4, 27] exploit the observation that as the SS sum c individuals, the central limit theorem guarantees that the distribution p ( s i | θ ) can be well-approximated by the Gaussian distribution N ( cµ g , c Σ g ) for large c [4]. Here, µ g and Σ g are the mean and covariance of an individual's SS. [3, 4, 27, 26] prescribe how to compute µ g and Σ g in closed form from the sampled θ parameters and moments of x and set the noise of the DP mechanism based on a sensitivity analysis. To approximate the posterior p ( θ | o ) , Markov Chain Monte Carlo (MCMC) sampling steps are needed. [4] propose to use Gibbs sampling, an algorithm that updates a group of parameters at a time and exploits conditional independence, for Bayesian linear regression models. [27] use the No-U-Turn [19] sampler and utilizes Hamiltonian dynamics to explore the parameter space more efficiently for generalized linear models. We describe the BLR Gibbs sampler adapted for our multi-party setting in Algo 1.

Algorithm 1 BLR Gibbs sampler [4] from noise-aware posterior p ( θ | O N = o N ) ∝ ∫ ∏ i ∈ N [ p ( o i | s i ) p ( s i | θ )] p ( θ ) d s 1 · · · d s n . The algorithm (repeatedly) sample the latent variables S i , ω and θ sequentially.

Require: Shared prior p ( θ ) of model parameters, prior p ( ω ) of data parameters, data quantity c i , shared perturbed SS realization o i , the Gaussian noise distribution of Z i for every party i ∈ N , number b of burn-in samples, number m of samples, Boolean parameter ( shared ) controlling if p ( x ) is the same across parties.

1: Sample the initial model parameters θ (0) from the prior p ( θ ) .

2: Sample the data prior parameters ω (0) from the prior p ( ω ) .

3: Compute the moments of X i based on ω .

4: for t = 1 , . . . , b + m do

5: for i = 1 , . . . , n do

6: Compute the normal approximation of p ( S i | θ ) , denoted as p N ( S i | θ ) , using the moments of X i .

7: Sample s ( t ) i from the product of two multivariate Gaussians p N ( S i | θ ) p ( o i | S i ) , which is also multivariate Gaussian.

8: if not shared then

10: end if

9: Use information from s ( t ) i and c i to perform conjugate update on p ( ω i ) to obtain p ( ω i | ( s ( t ) i , c i )) . Sample ω ( t ) i and compute the moments of X i .

11: end for

12: if shared then

14: end if

13: Use information from ( s ( t ) i , c i ) i ∈ N to perform conjugate update on p ( ω ) to obtain p ( ω | ( s ( t ) i , c i ) i ∈ N ) . Sample ω ( t ) and compute the moments of X i .

15: Use ( s ( t ) i , c i ) i ∈ N to perform conjugate update on p ( θ ) to obtain p ( θ | ( s ( t ) i , c i ) i ∈ N ) .

16: Sample θ ( t ) from p ( θ | ( s ( t ) i , c i ) i ∈ N ) .

17: if t > b then

18: Append θ ( t ) to Θ .

19: end if

20: end for

21: return Θ

B Key Differences with Existing Data Valuation, Collaborative ML, and DP/FL Works

FL

[34]

Data Valuation

[18, 25, 59]

DP Mechanisms

[1, 4, 27, 56]

◀

[47],

[55, 58]

▼

[20],

[26, 36]

▶

⋆

Incentives for Data Sharing

Higher reward for contributing more valuable data (fairness across parties, individual rationality, etc.)

Collaboratively Trained

Model

Each party receives a model decentrally trained with everyone's data

Differentially Private

Even to the mediator

Figure 5: Our work, ⋆ , uniquely satisfies all 3 desiderata. When parties share information computed from their data, we ensure that every party has at least its required DP w.r.t. the mediator, receives a collaboratively trained model, and receives a higher reward for sharing higher-quality data than the others.

It is not trivial to (i) add DP to ◀ while simultaneously enforcing a privacy-valuation trade-off, (ii) add data sharing incentives to ▼ (i.e., design valuation functions and rewards), and (iii) achieve ▶ as access to a party's dataset (or a coalition's datasets) is still needed for its valuation in [57].

Difference with existing data valuation and collaborative ML works considering incentives. Our work aims to additionally (A) offer parties assurance about privacy but (B) deter them from selecting excessive privacy guarantees. We achieve (A) by ensuring differential privacy (see definitions in App. A.2) through only collecting the noisier/perturbed version of each party's sufficient statistics (see App. A.1). To achieve (B), we must assign a lower valuation (and reward) to a noisier SS. Our insight is to combine noise-aware inference (that computes the posterior belief of the model parameters given the perturbed SS) with the Bayesian surprise valuation function. Lastly, (C) we propose a mechanism to generate model rewards (i.e., posterior samples of the model parameters) that attain the target reward value and are similar to the grand coalition's model.

Difference with federated learning and differential privacy works. Existing FL works have covered learning from decentralized data with DP guarantees. However, these works may not address the question: Would parties want to share their data? How do we get parties to share more to maximize the gain from the collaboration? Our work aims to address these questions and incentivize (A) parties to share more, higher-quality data and (B) select a weaker DP guarantee. To achieve (A), it is standard in data valuation methods [18, 25, 46] to use the Shapley value to value a party relative to the data of others as it considers a party's marginal contribution to all coalitions (subsets) of parties. This would require us to construct and value a trained model for each coalition C ⊆ N : To ease aggregation (and to avoid requesting more information or incurring privacy costs per coalition), we consider

sufficient statistics (see App. A.1). To achieve (B), we want a valuation function that provably ensures a lower valuation for a stronger DP guarantee. Our insight is to combine noise-aware inference (that computes the posterior belief of the model parameters given perturbed SS) with the Bayesian surprise valuation function. Lastly, like the works of [47, 51], (C) we generate a model reward that attains a target reward value (which parties can use for future predictions). Our model reward is in the form of posterior samples of the model parameters instead. We propose a new mechanism to control/generate model rewards that work using SS and preserve similarity to the grand coalition's model.

Fig. 5 shows how our work in this paper fills the gap in the existing works.

C Characteristic/Valuation Function

C.1 Proofs of properties for valuation function

In this section, we will use the random variable notations defined in App. A. Moreover, we abbreviate the set of perturbed SS random variables corresponding to a coalition C of parties as O C ≜ { O i } i ∈ C . Let H ( a ) denote the entropy of the variable a .

Relationship between KL divergence and information gain.

Party monotonicity (V2). Consider two coalitions C ⊂ C ′ ⊆ N . By taking an expectation w.r.t. random vector O C ′ ,

and

Then, E o C ′ ∼ O C ′ [ v C ′ ] > E o C ′ ∼ O C ′ [ v C ] as conditioning additionally on O C ′ \ C should not increase the entropy (i.e., H ( θ | O C , O C ′ \ C ) ≤ H ( θ | O C ) ) due to the 'information never hurts" bound for entropy [10].

θ

S

i

O

i

Z

1

i

O

s

i

Z

2

i

S

C

\{

i

}

O

C

\{

i

}

Privacy-valuation trade-off (V3). Let ϵ s i < ϵ i , and Z 1 i and Z 2 i be independent Gaussian distributions with mean 0 and, respectively, variance a i /ϵ i and ( a i /ϵ s i ) -( a i /ϵ i ) > 0 where a i ≜ 0 . 5 λ ∆ 2 2 ( g ) , function g computes the exact SS s i from local dataset D i , and ∆ 2 ( g ) denotes its ℓ 2 -sensitivity. Adding Z 1 i to S i will ensure ( λ, ϵ i ) -DP while adding both Z 1 i and independent Z 2 i to S i is equivalent to adding Gaussian noise of variance a i /ϵ s i to ensure ( λ, ϵ s i ) -DP. 18 From the graphical model

̸

in Fig. 6 and the Markov chain θ → O i → O s i , the following conditional independence can be observed: θ ⊥ ⊥ O s i | O i . By the data processing inequality , no further processing of O i , such as the addition of noise, can increase the information of θ . Formally, I ( θ ; O i ) ≥ I ( θ ; O s i ) . Simultaneously, θ ̸ ⊥ ⊥ O i | O s i . Hence, I ( θ ; O i ) = I ( θ ; O s i ) ⇒ ( I ( θ ; O i ) > I ( θ ; O s i )) .

To extend to any coalition C containing i , by the chain rule of mutual information,

As conditional independence θ ⊥ ⊥ O s i | O i , O C \{ i } and dependence θ ̸ ⊥ ⊥ O i | O s i , O C \{ i } still hold, I ( θ ; O s i | O i , O C \{ i } ) = 0 and I ( θ ; O i | O s i , O C \{ i } ) > 0 , respectively. It follows from the above expression that I ( θ ; O C ) > I ( θ ; O s i , O C \{ i } ) , which implies E O C [ v C ] > E O C \{ i } ,O s i [ v s C ] . For future work, the proof can be extended to other DP mechanisms.

C.2 Proof of Remark in Sec. 3

Let the alternative valuation of a coalition C be v ′ C ≜ D KL ( p ( θ | o N ); p ( θ )) -D KL ( p ( θ | o N ); p ( θ | o C )) . Then, v ′ ∅ = 0 and v ′ N = D KL ( p ( θ | o N ); p ( θ )) . It can be observed that

Unlike v C , v ′ C may be negative.

Unlike v N , v ′ N is guaranteed to have the highest valuation as the minimum KL divergence D KL ( p ( θ | o N ); q ( θ )) is 0 only when q ( θ ) = p ( θ | o N ) . This is desirable when we want the grand coalition to be more valuable than the other coalitions but odd when we consider the non-private posterior q ( θ ) = p ( θ | s N ) : Intuitively, the model computed using s N should be more valuable using v ′ than that computed using the perturbed SS o N .

By taking an expectation w.r.t. o N ,

In equality (i) above, we multiply both the numerator and denominator within the log term by p ( o N \ C | o C ) and consider the expectation of the joint distribution since by the chain rule of probability, p ( θ, O N \ C | o C ) = p ( O N \ C | o C ) p ( θ | O N \ C , o C ) . Equality (ii) is due to the definition of conditional mutual information.

C.3 KL Estimation of Valuation Function

KL estimation is only a tool and not the focus of our work. Our valuation will become more accurate and computationally efficient as KL estimation tools improve.

Recommended - nearest-neighbors [45, 54]. Given Θ post and Θ prior which consists of m samples of θ (with dimension d ) from, respectively, the posterior p ( θ | o C ) and prior p ( θ ) , we estimate the KL divergence as

where δ post k ( θ ) is the distance of the sampled θ to its k -th nearest neighbor in Θ post (excluding itself) and δ prior k ( θ ) is the distance of the sampled θ to the k -th nearest neighbor in Θ prior .

The number k of neighbors is tunable and analyzed in the follow-up work of [49]. As the number m of samples increases, the bias and variance decrease. The convergence rate is analysed by [66]. Moreover, the estimate converges almost surely [45] and is consistent [54] for independent and identically distributed (i.i.d.) samples . Furthermore, as the KL divergence is invariant to metric reparameterizations, the bias can be reduced by changing the distance metric [42, 54].

To generate i.i.d. samples, we suggest the usage of the NUTS sampler or thinning (keeping only every t -th sample). We observe that if the samples from θ | o C are non-independent, i.e., correlated and close to the previous sample, we may underestimate its distance to the k -th distinct neighbor in θ | o C , δ post k ( θ ) , and thus overestimate the KL divergence. This is empirically verified in Table 2. We have also observed that the KL divergence may be underestimated when the posterior is concentrated at a significantly different mean from the prior.

Recommended for large ϵ - approximate p ( θ | o C ) using maximum likelihood distribution from the p ( θ ) 's exponential family. When a small noise is added to ensure weak DP, we can approximate p ( θ | o C ) with a distribution q from the same exponential family as p ( θ | s C ) . We can (i) determine q 's parameters via maximum likelihood estimation (MLE) from the Gibbs samples 19 and (ii) compute the KL divergence in closed form.

However, the KL estimate is inaccurate (i.e., large bias) when the distribution q is a poor fit for the posterior p ( θ | o C ) . Future work can consider using normalizing flows as q to improve the fit, reduce the estimation bias, and work for a larger range of DP guarantees ϵ . However, this KL estimation method may be computationally slow and risks overfitting.

Probabilistic Classification. Let the binary classifier f : Θ → [0 , 1] (e.g., a neural network) discriminate between samples from two densities q 1 ( θ ) (here, the posterior p ( θ | o C ) ) and q 0 ( θ ) (here, the prior p ( θ ) ) and output the probability that θ comes from q 1 ( θ ) . Concretely, we label the m samples from q 1 ( θ ) and q 0 ( θ ) with y = 1 and y = 0 , respectively. By Bayes' rule, the density ratio is

Optimizing a proper scoring rule such as minimizing the binary cross-entropy loss should return the Bayes optimal classifier f ∗ ( θ ) = p ( y = 1 | θ ) . The KL estimate is then computed as the mean log-density ratio over samples from q 1 ( θ ) . As the log-density ratio is sigmoid -1 ( p ( y = 1 | θ )) , when f is a neural network with sigmoid as the last activation layer, we can use the logits before activation directly.

However, with only limited finite samples m and a large separation between the distributions q 1 and q 0 , the density ratio and KL estimate may be highly inaccurate [8]: Intuitively, the finite samples may be linearly separable and the loss is minimized by setting the logits of samples from q 1 ( θ ) (hence KL) to infinity (i.e., classify it overconfidently with probability 1 ). As the separation between the distributions q 1 and q 0 increases, exponentially more training samples may be needed to obtain samples between q 1 and q 0 [8]. Moreover, as training may not produce the Bayes optimal classifier, there is also an issue of larger variance across runs.

D Reward Scheme for Ensuring Incentives

D.1 Incentives based on Coalition Structure

Instead of assuming the grand coalition N form, we can consider the more general case where parties team up and partition themselves into a coalition structure CS . Formally, CS is a set of coalitions such that ⋃ C ∈ CS C = N and C ∩ C ′ = ∅ for any C, C ′ ∈ CS and C = C ′ . The following incentives are modified below:

̸

P2 For any coalition C ∈ CS , there is a party i ∈ C whose model reward is the coalition C 's posterior, i.e., q i ( θ ) = p ( θ | o C ) . It follows that r i = v C as in R2 of [47].

P5 Among multiple model rewards q i ( θ ) whose value r i equates the target reward r ∗ i , we secondarily prefer one with a higher similarity r ′ i,C = -D KL ( p ( θ | o C ); q i ( θ )) to the coalition's posterior p ( θ | o C ) where i ∈ C .

D.2 Fairness Axioms

The fairness axioms from the work of [47] are reproduced below:

F1 Uselessness. If including the data or sufficient statistic of party i does not improve the quality of a model trained on the aggregated data of any coalition (e.g., when D i = ∅ , c i = 0 ), then party i should receive a valueless model reward: For all i ∈ N ,

F2 Symmetry. If including the data or sufficient statistic of party i yields the same improvement as that of party j in the quality of a model trained on the aggregated data of any coalition (e.g., when D i = D j ), then they should receive equally valuable model rewards: For all i, j ∈ N s.t. i = j ,

F3 Strict Desirability [33]. If the quality of a model trained on the aggregated data of at least a coalition improves more by including the data or sufficient statistic of party i than that of party j , but the reverse is not true, then party i should receive a more valuable model reward than party j : For all i, j ∈ N s.t. i = j ,

F4 Strict Monotonicity. If the quality of a model trained on the aggregated data of at least a coalition containing party i improves (e.g., by including more data of party i ), ceteris paribus , then party i should receive a more valuable model reward than before: Let { v C } C ∈ 2 N and { ˜ v C } C ∈ 2 N denote any two sets of values of data over all coalitions C ⊆ N , and r i and ˜ r i be the corresponding values of model rewards received by party i . For all i ∈ N ,

D.3 Remark on Rationality

Let v s i denote the Bayesian surprise party i 's exact SS s i elicits from the prior belief of model parameter. We define Stronger Individual Rationality (SIR, the strengthened version of P4) as: each party should receive a model reward that is more valuable than the model trained on its exact SS alone: ∀ i ∈ Nr ∗ i ≥ v s i .

We consider two potential solutions to achieve stronger individual rationality and explain how they fall short.

Each party i declares the value v s i and the mediator selects a smaller ρ to guarantee SIR. SIR is infeasible when the grand coalition's posterior is less valuable than party i 's model based on exact SS, i.e., v N < v s 2 . In Fig. 2, we observe that when party 2 selects a small ϵ 2 , v N is less than v s 2 which can approximated by v 2 under large ϵ 2 , i.e., the right end point of the blue line. Instead, SIR is only empirically achievable when party 2 and others select a weaker DP guarantee as in Fig. 10 in App. H.5. Note that our work only incentivizes weaker DP guarantees and does not restrict parties' choice of DP guarantees.

̸

The mediator should reward party i with perturbed SS t i j (for Sec. 5.1) or κ i o j , κ i c j , κ i Z j (for Sec. 5.2) for every other party j = i . Party i can then combine these with its exact s i to guarantee SIR.

This approach achieves SIR at the expense of truthfulness. As party i perturbed SS o i is not used to generate its own model reward, party i may be less deterred (hence more inclined) to submit less informative or fake SS.

SIR is not needed when each party prefers training a DP model even when alone. SIR may be desired in other scenarios. However, our approach does not use alternative solutions to satisfy SIR as we prioritize incentivizing parties to (i) truthfully submit informative perturbed SS that they would use for future predictions, while (ii) not compromising for weak DP guarantees.

̸

̸

E Details on Reward Control Mechanisms

In the subsequent proofs, any likelihood p κ i ( · ) should be interpreted as [ p ( · )] κ i : We only raise likelihoods (of data conditioned on model parameters) to the power of κ i .

E.1 Likelihood Tempering and Scaling SS Equivalence Proof

Let g denote the function that maps any data point d l or dataset D k to its sufficient statistic. For any data point d l , we assume that the data likelihood p ( d l | θ ) is from an exponential family with natural parameters θ and sufficient statistic g ( d l ) . The data likelihood p ( d l | θ ) can be expressed in its natural form:

where a · b ≜ a ⊤ b denotes the dot product between two vectors.

Next, we assume that p ( θ ) is the conjugate prior 20 for p ( D k | θ ) with natural parameters η and the sufficient statistic mapping function T : θ → [ θ ⊤ , -A ( θ ) ] ⊤ . Then, for c k data points which are conditionally independent given the model parameters θ ,

where C ( η ) is chosen such that the distribution is normalized.

Substituting the above SS formulae into (1), the normalized posterior distribution (after tempering the likelihood) is

where C ′ ( η ) is chosen such that the distribution is normalized.

Thus, tempering the likelihood by κ i is equivalent to scaling the SS g ( D k ) and data quantity c k . We additionally proved that the normalized posterior can be obtained from the scaled SS and data quantity via the conjugate update.

Bayesian Linear Regression (BLR). The Bayesian linear regression model was introduced in App. A.1 To recap, BLR model parameters θ consists of the weight parameters w ∈ R w and the noise variance σ 2 . BLR models the relationship between the concatenated output vector y and the design matrix X as y = Xw + N (0 , σ 2 I ) . The tempered likelihood

only depends on the scaled sufficient statistics κ i ( y ⊤ y , X ⊤ y , X ⊤ X ) . When the prior p ( θ ) follows the conjugate normal inverse-gamma distribution, the power posterior can be obtained from the scaled SS and data quantity via the conjugate update.

Generalized Linear Model (GLM). App. A.1 introduces GLMs and states that the polynomial approximation to the GLM mapping function is an exponential family model. Tempering the GLM likelihood function by κ i is equivalent to scaling the GLM mapping function by κ i and can achieved by scaling the polynomial approximate SS by the same factor.

E.2 Smaller Tempering Factor Decreases Reward Value Proof

The KL divergence between two members of the same exponential family with natural parameters η and η ′ , and log partition function B ( · ) is given by ( η -η ′ ) ⊤ ∇ B ( η ) -B ( η ) + B ( η ′ ) [41]. To ease notational overload, we abuse some existing ones, which only apply in this subsection, by letting s N ≜ ∑ k ∈ N s k and c N ≜ ∑ k ∈ N c k . Let η ′ and η be the natural parameters of the prior and the normalized tempered posterior distribution (used to generate a model reward with value r i ), respectively. Then, η = η ′ + κ i [ s ⊤ N , c N ] ⊤ . For κ i ∈ [0 , 1] , the derivative of r i w.r.t. κ i is non-negative:

As B ( η ) is convex w.r.t. η , the second derivative 21 ∇ 2 B ( η ) is positive semi-definite, so [ s ⊤ N , c N ] ∇ 2 B ( η ) [ s ⊤ N , c N ] ⊤ ≥ 0 .

Hence, for κ i ∈ [0 , 1] , the KL divergence is non-decreasing as κ i increases to 1 . In other words, as κ i shrinks towards 0 , the KL divergence is decreasing; equality only holds when the variance of the SS is 0 .

E.3 Implementation of Reward Control Mechanisms

This subsection introduces how to obtain the model reward q i ( θ ) for each party i in Sec. 5.

Update for noise addition (varying τ i ). We update the inputs to Algorithm 1 for BLR or the No-U-Turn sampler for GLM. To generate party i 's posterior samples, for every party k ∈ N , the algorithm use the further perturbed SS t i k instead of the perturbed SS o k . Moreover, the algorithm consider the total DP noise Z k + N ( 0 , 0 . 5 λ ∆ 2 2 ( g k ) τ i I ) instead of only the noise Z k added by party k .

Update for likelihood tempering (varying κ i ). To generate party i 's posterior samples, for every party k ∈ N , Use κ i c k , κ i o k , and p ( κ i Z k ) as the inputs to Algorithm 1 for BLR or the No-U-Turn [19] sampler for GLM instead. Scaling the perturbed SS would affect the sensitivity of party k 's submitted information and the DP noise needed.

F Time Complexity

Algorithm 2 An overview of our collaborative ML problem setup. The computational complexity is given in App. F.

Require: Rényi DP λ parameter, Noise-aware inference algorithm, Shared prior p ( θ ) of model parameters and prior p ( ω ) of data parameters, ρ -Shapley fairness scheme parameter.

// Party's actions (ensure DP) 1: for each party i ∈ N do 2: Compute exact SS s i from dataset D i . 3: Choose DP guarantee ( λ, ϵ i ) -Rényi DP.

4: Sample z i from the Gaussian distribution p ( Z i ) = N ( 0 , 0 . 5 ( λ/ϵ i ) ∆ 2 2 ( g ) I ) .

5: Compute perturbed SS o i ≜ s i + z i .

6: Submit (i) number c i ≜ |D i | of data points in its dataset D i , (ii) perturbed SS o i and (iii) Gaussian distribution p ( Z i ) to the mediator.

7: end for

// Mediator's actions

// 1. Compute valuation of perturbed SS needed for Shapley value. The choice of v ensures a privacy-valuation trade-off.

8: Draw m samples from p ( θ ) .

9: for each coalition C ⊆ N do

10: Draw m samples from the posterior p ( θ | o C ) by applying the noise-aware inference algorithm. The algorithm requires the perturbed SS o C ≜ { o i } i ∈ C , data quantities { c i } i ∈ C and noise distributions { Z i } i ∈ C .

11: Compute v C by using the nearest-neighbors method [45] to estimate the KL divergence D KL ( p ( θ | o C ); p ( θ )) from the samples.

12: end for

// 2. Decide the target reward values using ρ -Shapley value [47] which ensure efficiency (P2), fairness (P3), rationality (P4) and control group welfare (P6).

13: for each party i ∈ N do

14: Compute Shapley value ϕ i = (1 /n ) ∑ C ⊆ N \ i [ ( n -1 | C | ) -1 ( v C ∪{ i } -v C ) ] .

15: end for

16: Identify the maximum Shapley value ϕ ∗ = max k ∈ N ϕ k .

17: for each party i ∈ N do

18: Compute ρ -Shapley fair target reward r ∗ i for party i using the formula r ∗ i = v N × ( ϕ i /ϕ ∗ ) ρ

19: end for

// 3. Generate model reward q i ( θ ) with value r i = r ∗ i that preserves similarity (P5) with the grand coalition's model and privacy for others (P1).

20: for each party i ∈ N do

21: Initialize Kr = () .

22: while True do

23: Select κ i ∈ [0 , 1] using a root finding algorithm and Kr .

24: Draw m samples from the normalized posterior q i ( θ ) (Eq. 1 by applying the noise-aware inference algorithm. Use the scaled perturbed SS { κ i o i } i ∈ N , data quantities { κ i c i } i ∈ N and noise distributions { κ i Z i } i ∈ N .

25: Compute the reward value r i by using the nearest-neighbors method [45] to estimate the KL divergence D KL ( q i ( θ ); p ( θ )) from the samples.

26: if attained reward value r i = r ∗ i then

27: Reward party i with the m posterior samples from q i ( θ ) .

28: break

29: end if

30: Update Kr ← Kr +(( κ i , r i ) , )

31: end while

32: end for

The main steps of our scheme are detailed in Algo. 2 and the time complexity of the steps are as follows:

Local SS s i computation (Line 2 in Algo 2). Party i will need to sum the SS for its c i data points. Subsequent steps will not depend on the number c i of data points. The (approximate) SS is usually an O ( d 2 ) vector where d is the number of regression model features. Therefore, this step incurs O ( c i d 2 ) time .

Perturbed SS o i computation (Lines 4-5 in Algo 2). Each party will need to use the Gaussian mechanism to perturb s i . Therefore, this step incurs O ( d 2 ) time .

A

Valuation of Perturbed SS (Sec. 3). The valuation of o N requires us to draw m posterior samples of model parameters using DP noise-aware inference (refer to App. A.3 and the cited references for the exact steps). As the methods of [27] and [4] incur O ( md 4 ) time for a single party, inference based on Fig. 4 will take n times longer to consider n parties. KL estimation using k -nearest neighbor will incur O ( m log( m )dim( θ )) time to value multiple (scaled) perturbed SS. Therefore, valuation incurs O ( nmd 4 + m log( m ) dim( θ )) time .

Deciding target reward value r ∗ i for every i ∈ N (Sec. 4, Lines 9-19 in Algo 2)). Computing the Shapley values exactly involves valuing o C for each subset C ⊆ N , hence, repeating Step A O (2 n ) time. When the number of parties is small (e.g., < 6 ), we can compute the Shapley values exactly. For larger n , we can approximate the Shapley values ( ϕ i ) i ∈ N with bounded ℓ 2 -norm error using O ( n (log n ) 2 ) samples [25, 53]. Moreover, the value of different coalitions can be computed in parallel . Therefore, this step incurs O (2 n ) or O ( n (log n ) 2 ) times the time in Step A .

Solving for κ i to generate model reward (Sec. 5.2, Lines 21-31 in Algo 2). During root-finding, the mediator values different model rewards q i ( θ ) generated by scaling the perturbed SS o k , data quantity c k and DP noise distribution Z k of each party k ∈ N by different κ i , hence, repeats Step A. As we are searching for the root in a fixed interval [0 , 1] and to a fixed precision, Step A is repeated a constant (usually < 10 ) number of times. Therefore, this step incurs O ( nmd 4 + m log( m ) dim( θ )) time per party .

The mediator can further reduce the number of valuation of model rewards (repetitions of Step A) by using the tuples of ( κ i , r i ) obtained when solving for κ i to narrow the root-finding range for other parties after i .

Therefore, the total incurred time depends on the number of valuations performed in Step A. The time complexity may vary for other inference and KL estimation methods.

G Comparison of Reward Control Mechanisms via Noise Addition (Sec. 5.1) vs. Likelihood Tempering (Sec. 5.2)

See Fig. 7.

2

1

0

1

2

-

2

-

1

0

1

2

p

(

w

1

, w

2)

p

(

w

1

, w

2

|

oN

)

p

(

w

1

, w

2

|

t

2

N

)

pκ

2

(

w

1

, w

2

|

oN

)

-

-

H Experiments

The experiments are performed on a machine with Ubuntu 20 . 04 LTS, 2 × Intel Xeon Gold 6230 ( 2 . 1 GHz) without GPU. The software environments used are Miniconda and Python. A full list of packages used is given in the file environment.yml attached.

H.1 Experimental Details

Synthetic BLR (Syn). The BLR parameters θ consist of the weights for each dimension of the 2 D dataset, the bias, and the variance σ 2 . The normal inverse-gamma distribution used (i) to generate the true regression model weights, variance, and a 2 D dataset and (ii) as our model prior is as follows: σ 2 ∼ InvGamma ( α 0 = 5 , β 0 = 0 . 1) where α 0 and β 0 are, respectively, the inverse-gamma shape and scale parameters, and w | σ 2 ∼ N ( 0 , σ 2 Λ -1 0 ) where Λ 0 = 0 . 025 I .

We consider three parties 1 , 2 , and 3 with c 0 = 100 , c 1 = 200 , and c 2 = 400 data points, respectively. We fix ϵ 1 = ϵ 3 = 0 . 2 and vary ϵ 2 from the default 0 . 1 . As ϵ 2 increases (decreases), party 2 may become the most (least) valuable. We allow each party to have a different Gaussian distribution p ( x i ) by maintaining a separate conjugate normal inverse-Wishart distribution p ( ω i = ( µ ω,i , Σ ω,i )) for each party. We set the prior Σ ω,i ∼ W -1 ( ψ 0 = I , ν 0 = 50) where ψ 0 and ν 0 are the scale matrix and degrees of freedom (i.e., how strongly we believe the prior), respectively. Then, µ ω,i ∼ N ( 0 , Σ ω,i ) . The ℓ 2 -sensitivity is estimated using [26]'s analysis based on the norms/bounds of the dataset.

One posterior sampling run generates 16 Gibbs sampling chains in parallel. For each chain, we discard the first 10000 burn-in samples and draw m = 30000 samples. To reduce the closeness/correlation between samples which will affect the nearest-neighbor-based KL estimation, we thin and only keep

every 16 -th sample and concatenate the thinned samples across all 16 chains. For the experiment on reward control mechanisms, we use 5 independent runs of posterior sampling and KL estimation.

Californian Housing dataset (CaliH). As the CaliH dataset may contain outliers, we preprocess the 'public' dataset ( 60% of the CaliH data) by subtracting the median and scaling by the interquartile range for each feature. We train a neural network (NN) of 3 layers with [48 , 24 , 6] hidden units and the rectified linear unit (ReLU) as the activation function to minimize the mean squared error, which we will then use as a 'pre-trained NN'. The outputs of the last hidden layer have 6 features used as the inputs for BLR. We intentionally reduce the number of features in the BLR model by adding more layers to the pre-trained NN and reduce the magnitude of the BLR inputs by adding an activation regularizer on the pre-trained NN hidden layers (i.e., ℓ 2 penalty weight of 0 . 005 ). These reduce the computational cost of Gibbs sampling/KL estimation and the ℓ 2 -sensitivity of the inputs to BLR (hence the added DP noise), respectively. We also add a weights/bias regularizer with an ℓ 2 penalty weight of 0 . 005 for the last layer connected to the outputs. Lastly, we standardize the outputs of the last hidden layer to have zero mean and unit variance.

We preprocess the private dataset for valuation and the held-out validation set (an 80 -20 split) using the same pre-trained NN/transformation process. To reduce the sensitivity and added DP noise, we filter and exclude any data point with a z -score > 4 for any feature. There are 6581 training data points left. We divide the dataset randomly among 3 parties such that parties 1 , 2 , and 3 have, respectively, 20% , 30% and 50% of the dataset and ϵ 1 = ϵ 3 = 0 . 2 while ϵ 2 is varied from the default 0 . 1 .

The BLR parameters θ consist of the weights for each of the 6 features, the bias, and the variance σ 2 . We assume θ has a normal inverse-gamma distribution and set the prior as follows. The prior depends on the MLE estimate based on the public dataset, and we assume it has the same significance as n 0 = 10 data points. Hence, we set σ 2 ∼ InvGamma ( α 0 = n 0 / 2 , β 0 = n 0 / 2 × MLE estimate of σ 2 ) and w | σ 2 ∼ N ( 0 , σ 2 ( n 0 x ⊤ x ) -1 ) .

We assume that each party has the same underlying Gaussian distribution for p ( x ) and maintain only one conjugate normal inverse-Wishart distribution p ( ω = ( µ ω , Σ ω )) shared across parties. We initialize the prior p ( ω ) to be weakly dependent on the prior dataset [39]. The ℓ 2 -sensitivity is estimated using [26]'s analysis based on the norms/bounds of the private transformed dataset.

One posterior sampling run generates 16 Gibbs sampling chains in parallel. For each chain, we discard the first 10000 burn-in samples and draw m = 30000 samples. To reduce the closeness/correlation between samples which will affect the nearest-neighbor-based KL estimation, we thin and only keep every 16 -th sample and concatenate the thinned samples across all 16 chains. For the experiment on reward control mechanisms, we use 5 independent runs of posterior sampling and KL estimation.

PIMA Indian Diabetes classification dataset (Diab). This dataset has 8 raw features such as age, BMI, number of pregnancies, and a binary output variable. Patients with and without diabetes are labeled y = 1 and y = -1 , respectively. We split the training and the validation set using an 80 -20 split. There are 614 training data points. There are 35 . 6% and 31 . 8% of patients with diabetes in the training and validation sets, respectively. Hence, random guessing would lead to a cross-entropy loss of 0 . 629 .

We preprocess both sets by (i) subtracting the training set's median and scaling by the interquartile range for each feature, (ii) using principal component analysis (PCA) to select the 4 most important components of the feature space to be used as new features, and lastly, (iii) centering and scaling the new features to zero mean and unit variance. To reduce the effect of outliers and the ℓ 2 -sensitivity, we clip each training data point's feature values at ± 2 . 2 .

We divide the 614 training data points such that parties 1 , 2 , and 3 have, respectively, 20% , 30% , and 50% of the dataset and ϵ 1 = ϵ 3 = 0 . 2 while ϵ 2 is varied from the default 0 . 1 . We compute the approximate SS [21] and perturb them for each party to achieve the selected ϵ i [27]. The ℓ 2 -sensitivity is also estimated based on the dataset.

We consider a Bayesian logistic regression model, and its parameters θ consist of the bias and the weights for each of the 4 features. Like that of [27], we set an independent standard Gaussian prior for θ but rescale it such that the squared norm ∥ θ ∥ 2 2 has a truncated Chi-square prior with d = 4 degrees

of freedom. Truncation prevents sampling θ with a norm larger than 2 . 5 times the non-private/true setting's θ ∗ squared norm during inference.

We assume each party has the same distribution for p ( x ) . Our data prior p ( x ) has mean 0 and covariance Σ = diag ( ι ) Ω diag ( ι ) where ι ∼ N ( 1 , I ) , ι ∈ [0 , 2] , and Ω follows a LewandowskiKurowicka-Joe prior LKJ (2) .

We use the No-U-Turn [19] sampler. We run 25 Markov chains with 400 burn-in samples and draw m = 2000 samples with a target Metropolis acceptance rate of 0 . 86 . We discard chains with a low Bayesian fraction of missing information (i.e., < . 3 ) and split the concatenated samples across chains into 5 groups to estimate KL divergence. As sampling is slower and the generated samples tend to be less correlated, we can use fewer samples.

Remark. For the CaliH dataset, the preprocessing is based on the 'public' dataset, but for the Diab dataset, the preprocessing (i.e., standardization, PCA) is based on the private, valued dataset. We have assumed that the data is preprocessed. However, in practice, before using our mechanism, the parties may have to reserve/separately expend some privacy budget for these processing steps. The privacy cost is ignored in our analysis of the privacy-valuation trade-off.

KL estimation. We estimate KL divergence using the k -nearest-neighbor-based KL estimator [45]. To reduce the bias due to the skew of the distribution, we apply a whitening transformation [54] where each parameter sample is centered and multiplied by the inverse of the sample covariance matrix based on all samples from θ and θ | o . As a default, we set k = 4 and increase k until the distance to the k -th neighbor is non-zero.

H.2 Utility of Model Reward

The mean negative log probability (MNLP) on a test dataset D ∗ given the perturbed SS o i is defined as follows:

We prefer MNLP over the model accuracy or mean squared error metric. MNLP additionally measures if a model is uncertain of its accurate predictions or overconfident in inaccurate predictions. In contrast, the latter metrics penalize inaccurate predictions equally and ignore the model's confidence (which is affected by the DP noise).

Regression. Approximating the predictive distribution, p ( y ∗ | x ∗ , o i ) , for test input x ∗ as Gaussian, the MNLP formula becomes

where µ ∗ and ̂ σ 2 ( x ∗ ) denote the predictive mean and variance, respectively. The first term penalizes large predictive variance while the second term penalizes inaccurate predictions more strongly when the predictive variance is small (i.e., overconfidence).

The predictive mean ̂ µ ( x ∗ ) is the averaged prediction of y ∗ (i.e., w ⊤ x ∗ , where w is part of the model parameters θ ) over all samples of the model parameters θ .

The predictive variance ̂ σ 2 ( x ∗ ) is computed using the variance decomposition formula, i.e., the sum of the averaged σ 2 (the unknown variance parameter within θ ) and the computed variance in predictions over samples, i.e., = m -1 ∑ m j =1 σ 2 j + ̂ µ 2 ( x ∗ ) -̂ µ ( x ∗ ) 2 .

Classification. Wecan estimate p ( y ∗ | x ∗ , o i ) , for test input x ∗ using the Monte Carlo approximation [39] and reusing the samples θ from p ( θ | o i ) . Concretely, p ( y = 1 | x ∗ , o i ) ≊ m -1 ∑ m j =1 σ ( θ ⊤ x ∗ ) . The MNLP is equivalent to the cross-entropy loss.

H.3 Baselines

To plot the figures in Sec. 6, the baseline DP and collaborative ML works must

work for similar models, i.e., Bayesian linear and logistic regression;

not use additional information to value coalitions and generate model rewards (to preserve the DP post-processing property); and

decide feasible model reward values and suggest how model rewards can be generated.

Work of [59]. Valuation by volume is model-agnostic (satisfying criteria 1). Each party i ∈ N can submit the noisy version of X ⊤ i X i with DP guarantees to the mediator who can sum them to value coalitions (satisfying criteria 2 ). The work does not propose a model reward scheme to satisfy criteria 3 .

Work of [47]. [47] only considered Bayesian linear regression (with known variance) and it is not straightforward to compute information gain on model parameters for Bayesian linear regression (with unknown variance) and Bayesian logistic regression. Thus, the work does not satisfy criteria 1 . For Bayesian linear regression (with known variance), each party i ∈ N can submit the noisy version of X ⊤ i X i with DP guarantees to the mediator who can sum them to value coalitions (satisfying criteria 2 ). The work proposed a model reward scheme which involves adding noise to the outputs y ( satisfying criteria 3 but has to be adapted to ensure DP).

DP-FL works. A promising approach is to use DP-FedAvg/DP-FedSGD [36] to learn any model parameters (satisfying criteria 1) in conjunction with FedSV [55] to value coalitions without additional information (satisfying criteria 2). However, to our knowledge, these works will not satisfy criteria 3 as they do not suggest how to generate model rewards of a target reward value.

As no existing work satisfies all criteria, we compare against (1) using non-noise-aware inference instead of noise-aware inference, all else equal (in App. H.5); and (2) an adapted variant of the reward control via noise addition (in Sec. 5.1, Sec. 6, and App. G).

H.4 Valuation Function

In Sec. 6, we only vary the privacy guarantee ϵ i of one party i . In this subsection, we will analyze how other factors such as the coverage of the input space and the number of posterior samples on the valuation v i .

Coverage of input space. We vary the coverage of the input space by only keeping those data points whose first feature value is not greater than the 25 , 50 , 75 , 100 -percentile. Across all experiments in Table 1, it can be observed that as the percentile increases (hence, data quantity and coverage improve), the surprise elicited by the perturbed SS o N increases in tandem with the surprise elicited by the exact SS s N .

Number of posterior samples. For a consistent KL estimator, the bias/variance of the KL estimator should decrease with a larger number of posterior samples.

Gibbs sampling. We compare the estimated surprise using various degrees of thinning (i.e., keeping only every t -th sample) to generate 30000 samples for the CaliH dataset. In Table 2, it can be observed that although the total number of samples is constant, the surprise differs significantly. Moreover,

as t increases, the surprise decreases at a decreasing rate and eventually converges. This may be because consecutive Gibbs samples are highly correlated and close, thus causing us to underestimate the distance to the k -th nearest-neighbors within θ | o N (see discussion in App. C.3). Increasing t reduces the correlation and closeness and better meets the i.i.d. samples assumption of the nearestneighbor-based KL estimation method [45].

NUTS logistic regression. After drawing 10000 samples for the Diab dataset using the default setting, we analyze how using a subset of the samples will affect the estimated surprise. In particular, we consider using (i) the first m samples or (ii) thinned m samples where we only keep every 10000 /m -th sample.

In Table 3, it can be observed that as the number m of samples increases, the standard deviation of the estimated surprise decreases. Moreover, there is no significant difference between using the first m samples or the thinned m samples. This suggests that the samples are sufficiently independent and thinning is not needed.

H.5 Additional Experiments on Valuation, Privacy-valuation Trade-off, and Privacy-reward Trade-off

-

2

-

1

0

1

log10

/epsilon1

2

2

4

φ

1

φ

2

φ

3

-

2

-

1

0

1

log10

/epsilon1

2

0

2

4

6

8

φ

1

φ

2

φ

3

-

2

-

1

0

1

log10

/epsilon1

2

1

2

3

φ

1

φ

2

φ

3

(a) Syn

(b) CaliH

(c) Diab

Shapley value. In Fig. 8, it can be observed that as party 2 weakens its privacy guarantee (i.e., ϵ 2 increases), its Shapley value ϕ 2 increases while other parties' Shapley values (e.g., ϕ 3 ) decrease.

When party 2 adds less noise to generate its perturbed SS o 2 , others add less value (i.e., make lower marginal contributions (MC)) to coalitions containing party 2 . Party 2 changes from being least valuable to being most valuable, even though it has more data than party 1 and less data than party 3 .

-

2

-

1

0

1

log10

/epsilon1

2

10

20

30

v

2

φ

2

r

2

vN

-

2

-

1

0

1

log10

/epsilon1

2

20

40

60

v

2

φ

2

r

2

vN

-

2

-

1

0

1

log10

/epsilon1

2

3

6

9

v

2

φ

2

r

2

vN

(a) Syn

(b) CaliH

(c)Diab

Without DP noise-aware inference. In Fig. 9a, it can be observed that as ϵ 2 increases, v i and ϕ i for party i = 2 do not strictly increase. In Figs. 9b-c, it can be observed that as ϵ 2 increases, v i and ϕ i for party i = 2 decrease instead. The consequence of non-noise-aware inference is undesirable for incentivization - party 2 unfairly gets a lower valuation and reward for using a weaker privacy guarantee, i.e., a greater privacy sacrifice. Moreover, when ϵ 2 is small (i.e., under a strong privacy guarantee), party 2 is supposed to be least valuable. However, the significantly different o 2 causes party 2 to have the highest valuation and be rewarded with the grand coalition N 's model (i.e., r i close to v N ) instead.

Lastly, we also observe that without DP noise-aware inference, the utility of the model reward is small. For example, the naive posterior for the Syn dataset results in an MNLP larger than 100 .

̸

Conditions for larger improvement in MNLP. In Fig. 3, it seems that the utility of party i = 2 's model reward measured by MNLP r cannot improve significantly over over that of its individually trained model when ϵ 2 is large. However, party i 's MNLP r can be improved by a larger extent when (i) any other party j = i selects a weaker privacy guarantee (i.e., a larger ϵ j ), thus improving the utility of the collaboratively trained model or (ii) party i and others have lower data quantity (i.e., smaller c k for all k ∈ N ) and are unable to individually train a model of high utility. Figs. 10a, 10b, and 10c are examples of (i), (ii), and (i+ii), respectively. In Fig. 10a, the MNLP N of grand coalition N 's collaboratively trained model is lower than that in Fig. 3a. In Fig. 10b, the MNLP i of party i 's model is higher due to less data. In these examples, we observe that a party can still gain a significant improvement MNLP i -MNLP r when ϵ i > 1 .

Condition (i) for a larger improvement in MNLP r is satisfied when the trade-off deters parties from selecting excessive DP guarantees, i.e., it incentivizes parties to select weaker DP guarantees that still meet their legal and customers' requirements. Condition (ii) should be satisfied in most real-life scenarios where a party wants to participate in collaborative ML and federated learning. The party (e.g., bank) is unable to achieve its desired utility with its individually trained model due to limited data and collaborates with others to unlock any improvement in the utility of a collaboratively trained model.

0

2

log10

/epsilon1

2

0

.

0

0

.

5

MNLP

N

r

i

=2

-

2

0

log10

/epsilon1

2

0

.

0

0

.

5

MNLP

N

r

i

=2

-

2

0

log10

/epsilon1

1

0

.

6

0

.

7

MNLP

i

=1

r

N

(a) Syn, larger

ϵ

j

(b) Syn, halved

{

c

k

}

k

∈

N

(c) Diab, larger

ϵ

k

&

halved

{

c

k

}

k

∈

N

Higher λ = 10 . In Fig. 11, the privacy-valuation, privacy-reward, and privacy-utility trade-offs are still observed when parties select a higher λ = 10 when enforcing the Rényi DP guarantee. Moreover, the utility of party 2 's model reward is higher (i.e., lower MNLP) than that of its individually trained model.

-

2

-

1

0

1

log10

/epsilon1

2

0

2

4

6

v

2

φ

2

r

2

vN

-

2

-

1

0

1

log10

/epsilon1

2

0

4

8

12

v

2

φ

2

r

2

vN

-

2

-

1

0

1

log10

/epsilon1

2

1

2

3

v

2

φ

2

r

2

vN

(a) Syn

(b) CaliH

(c) Diab

-

2

0

log10

/epsilon1

2

-

0

.

4

0

.

0

0

.

4

0

.

8

MNLP

N

r

i

=2

-

2

0

log10

/epsilon1

2

1

2

MNLP

N

r

i

=2

-

2

0

log10

/epsilon1

2

0

.

60

0

.

65

0

.

70

MNLP

i

=2

r

N

(d) Syn (MNLP vs.

ϵ

2

) (e) CaliH (MNLP vs.

ϵ

2

) (f) Diab (MNLP vs.

ϵ

2

)

H.6 Additional Experiments on Reward Control Mechanisms

For the CaliH dataset, there is a monotonic relationship between r i vs. both κ i and τ i , as shown in Fig. 12a. However, it can be observed from Figs. 12b-c that for the same attained reward value r i , adding scaled noise variance τ i will lead to a lower similarity r ′ i to the grand coalition N 's posterior p ( θ | o N ) and utility of model reward (higher MNLP r ) than tempering the likelihood by κ i .

0

5

10

log2

τi

3

6

9

ri

0

.

0

0

.

5

1

.

0

κi

5

10

ri

-

10

-

8

-

6

-

4

-

2

0

r

′

i

κi

τi

5

10

ri

1

.

0

1

.

5

MNLPr

κi

τi

(a)

r

i

vs.

κ

i

,

τ

i

(b)

r

′

i

vs.

r

i

(c) MNLP

r

vs.

r

i

For Diab dataset, there is a monotonic relationship between r i vs. both κ i and τ i , as shown in Fig. 13a. However, it can be observed from Fig. 13b-c that for the same attained reward value r i , tempering the likelihood by κ i leads to a higher similarity r ′ i to the grand coalition N 's posterior p ( θ | o N ) and utility of model reward (lower MNLP r ) than adding scaled noise variance τ i .

0

4

8

log2

τi

0

.

5

1

.

0

1

.

5

2

.

0

ri

0

.

0

0

.

5

1

.

0

κi

1

2

ri

-

1

0

r

′

i

τi

κi

1

2

ri

0

.

6

0

.

7

MNLPr

τi

κi

(a)

r

i

vs.

κ

i

,

τ

i

(b)

r

′

i

vs.

r

i

(c) MNLP

r

vs.

r

i

Problematic noise realization. Wewill show here and in Fig. 14a that some (large) noise realization can result in a non-monotonic relationship between the attained reward value r i vs. the scaled additional noise variance τ i . As a result, it is hard to bracket the smallest root τ i that solves for r i = r ∗ i (e.g., = 2 or = 3 ). Moreover, it can be observed from Figs. 14b-c that the model reward's posterior q i ( θ ) has a low similarity r ′ i to the grand coalition N 's posterior p ( θ | o N ) and a much higher MNLP r than the prior. This suggests that injecting noise does not interpolate well between the prior and the posterior. In these cases, it is not suitable to add scaled noise variance τ i and our reward control mechanism via likelihood tempering with κ i , is preferred instead.

0

7

14

log2

τi

2

3

4

5

ri

0

.

0

0

.

5

1

.

0

κi

2

4

ri

-

10

-

8

-

6

-

4

-

2

0

r

′

i

κi

τi

2

3

4

5

ri

0

2

4

6

8

MNLPr

κi

τi

Prior

(a)

r

i

vs.

κ

i

,

τ

i

(Non-monotonic in

τ

i

)

(b)

r

′

i

vs.

r

i

(c) MNLP

r

vs.

r

i

I Other Questions

Question 1: Are there any ethical concerns we foresee with our proposed scheme?

Answer: Our privacy-valuation trade-off (V3) should deter parties from unfetteredly selecting excessively strong DP guarantees. Parties inherently recognize the benefits of stronger DP guarantees and may prefer such benefits in collaboration out of overcaution, mistrust of others, and convenience. The trade-off counteracts (see Fig. 1) the above perceived benefits by explicitly introducing costs (i.e., lower valuation and quality of model reward). Consequently, parties will carefully select a weaker yet satisfactory privacy guarantee they truly need.

However, a potential concern is that parties may opt to sacrifice their data's privacy to obtain a higher-quality model reward. The mediator can alleviate this concern by enforcing a minimum privacy guarantee (i.e., maximum ϵ ) each party must select. The model rewards will preserve this minimum privacy guarantee due to P1. The mediator can also decrease the incentive by modifying v C .

Another potential concern is that if parties have data with significantly different quantity/quality/privacy guarantees, the weaker party k with fewer data or requiring a stronger privacy guarantee will be denied the best model reward (i.e., trained on the grand coalition's SS) and instead rewarded with one that is of lower quality for fairness. The mediator can alleviate the concern and at least ensure individual rationality (P4) by using a smaller ρ so that a weaker party k can obtain a higher-quality model reward with a higher target reward value r ∗ k .

Question 2: Is it sufficient and reasonable to value parties based on the submitted information { c i , o i , p ( Z i ) } i ∈ N instead of ensuring and incentivizing truthfulness? Would parties strategically declare other values to gain a higher valuation and reward?

Answer: An ideal collaborative ML scheme should additionally incentivize parties to be truthful and verify the authenticity of the information provided. However, achieving the 'truthfulness' incentive is hard and has only been tackled by existing works to a limited extent. Existing work cannot discern if the data and information declared are collected or artificially created (e.g., duplicated) and thus, this non-trivial challenge is left to future work. The work of [32] assigns and considers each client's reputation from earlier rounds, while the works of [29, 31] measure the correlation in parties' predictions and model updates. The work of [7] proposes a payment rule based on the log pointwise mutual information between a party's dataset and the pooled dataset of others. This payment rule guarantees that when all other parties are truthful (i.e., a strong assumption), misreporting a dataset with an inaccurate posterior is worse (in expectation) than reporting a dataset with accurate posterior. 22

Thus, like the works of [18, 17, 25, 40, 47, 51] and others, we value data as-is and leave achieving the 'truthfulness' incentive to future work. In practice, parties such as hospitals and firms will truthfully share information as they are primarily interested in building and receiving a model reward of high quality and may additionally be bound by the collaboration's legal contracts and trusted data-sharing platforms like Ocean Protocol [43]. For example, with the use of X-road ecosystem, 23 parties can maintain a private database which the mediator can query for the perturbed SS. This ensures the authenticity of the data (also used by the owner) and truthful computation given the uploaded private database.

Lastly, a party k who submits fake SS will also reduce its utility from the collaboration. Party k 's fake SS will affect the grand coalition's posterior of the model parameters given all perturbed SS and is also used to generate k 's model reward. As party k only receives posterior samples, k cannot replace the fake SS with its exact SS locally. As party k have to bear the consequences of the fake SS, it would be more likely to submit true information.

Question 3: Why do we only consider Bayesian models with SS?

Answer: See App A.1 for a background on SS. Our approach would also work for Bayesian models with approximate SS, such as Bayesian logistic regression, and latent features extracted by a neural network.

The exact SS s i captures all the information (i.e., required by the mediator) within party i 's dataset D i . Thus, the mediator can do valuation and generate model rewards from the perturbed SS { o i } i ∈ N without requesting more information from the parties. This limits the privacy cost and allows us to rely on the DP post-processing property.

In Sec. 3, the proof that Def. 3.1 satisfies a privacy-valuation trade-off (V3) uses the properties of SS.

Our work introduces privacy as an incentive and simultaneously offers a new perspective that excessive DP can and should be deterred by introducing privacy-valuation and privacy-reward trade-offs and accounting for the DP noise. We use Bayesian models with SS as a case study to show how the incentives and trade-offs can be achieved. It is up to the future work to address the non-trivial challenge of ensuring privacy-valuation and privacy-reward trade-offs for other models.

Question 4: Can alternative fair reward schemes be used in place of ρ -Shapley fair reward scheme [47]?

Answer: Yes, if they satisfy P3 and P4. For example, if the exchange rate between the perturbed SS quality and monetary payment is known, then the scheme of [40] can be used to decide the reward instead. Our work will still ensure the privacy-valuation trade-off and provide the mechanism to generate the model reward q i ( θ ) to attain any target reward value r ∗ i while preserving similarity to the grand coalition N 's model (P5).

Question 5: What is the difference between our work here and that of [47]?

Answer: We clearly outlined our contributions in bullet points at the end of the introduction section (Sec. 1) and in Fig. 1.

At first glance, our work seems to only add a new privacy incentive. However, as discussed in the introduction section (Sec. 1), privacy is barely considered by existing collaborative ML works and raises significant challenges. The open questions/challenges in [64]'s survey on adopting DP in gametheoretic mechanism design (see Sec. 7 . 1 therein) inspire us to ask the following questions:

How can DP and the aims of cooperative game theory-inspired collaborative ML be compatible? Will DP invalidate existing properties like fairness?

How should parties requiring a strong DP guarantee be prevented from unfairly and randomly obtaining a high-quality model reward?

We propose to enforce a provable privacy-valuation trade-off to answer the latter. The enforcement involves novelly selecting and combining the right valuation function and tools, such as DP noiseaware inference.

Additionally, we propose a new reward control mechanism that involves tempering the likelihood (practically, scaling the SS) to preserve similarity to the grand coalition's model (P5) and hence increase the utility of the model reward.

Question 6: Will a party with high-quality data (e.g., a large data quantity, less need for DP guarantee) be incentivized to participate in the collaboration?

Answer: From Fig. 3, it may seem that a rich party i with ample data and a weak privacy guarantee (i.e., large ϵ i ) has a lower utility of model reward to gain from the collaboration. However, it may still be keen on a further marginal improvement in the utility of its model reward (e.g., increasing the classification accuracy from 97% to 99% and predicting better for some sub-groups) and can reasonably expect a better improvement as other parties are incentivized by our scheme (through enforcing a privacy-valuation trade-off and fairness F4) to contribute more data at a weaker yet satisfactory DP guarantee (see App. H.5). Moreover, a rich party does not need to be concerned about others unfairly benefiting from its contribution as our scheme guarantees fairness through Shapley value. In Fig. 8, as a party selects a weaker DP guarantee (and all

else being held constant), the Shapley values of others, which determine their model rewards, decrease.

Question 7: What is the impact of varying other hyperparameters?

Answer: The work of [47] proposes ρ -Shapley fairness and theoretically and empirically show that any ρ > 0 guarantees fairness across parties and a smaller ρ will lead to a higher attained reward value r i for all other parties which do not have the largest Shapley value. These properties apply to our problem setup, and using a larger ρ will worsen/reduce r i and the utility of party i 's model reward q i ( θ ) measured by MNLP r . The work of [47] has empirically shown that the number of parties does not impact the scheme's effectiveness. However, it affects the time complexity to compute the exact and approximate SV.

More importantly, the extent to which party i can benefit from its contribution depends on the quantity/quality of its data relative to that of the grand coalition N (and the suitability of the model or informativeness of the prior).

Party i 's DP guarantee ϵ i is varied in Sec. 6 while the DP guarantee ϵ k of the other party k and its number c k of data points for k ∈ N are varied in App. H.5. The privacy order λ is varied in App. H.5. Across all experiments, we observe that the privacy-valuation trade-off holds. Moreover, when (i) a party i has lower-quality data in the form of fewer data points or smaller ϵ i , or (ii) another party k has higher-quality data such as a larger ϵ k , the improvement in the utility of its model reward over that of its individually trained model is larger.

Question 8: Can privacy be guaranteed by using secure multi-party computation and homomorphic encryption in model training/data valuation?

Answer: These techniques are designed to prevent direct information leakage and prevent the computer from learning anything about the data. However, as the output of the computation is correct, any mediator and collaborating party with access to the final model can query the model for predictions and infer private information/membership of a datum (indirect privacy leakage). In our work here, every party can access a model reward. Hence, the setup should prevent each party from inferring information about a particular instance in the data beyond what can be learned from the underlying data distribution through strong DP guarantees .

Question 9: In Sec. 4, we mention that (i) it is possible to have negative marginal contributions (i.e., v C ∪ i < v C ) in rare cases and (ii) adding some noise realizations may counter-intuitively create a more valuable model reward (e.g., r i > v N ). Why and what are the implications?

Answer: For our choice of valuation function via Bayesian surprise, the party monotonicity (V2) and privacy-valuation trade-off (V3) properties involve taking expectations, i.e., on average/in most cases , adding a party will not decrease the valuation (i.e., the marginal contribution is non-negative), and strengthening DP by adding more noise should decrease the reward value. However, in rare cases, (i) and (ii) can occur. We have never observed (i) in our experiments, but a related example of (ii) is given in Fig. 14a: A larger τ i surprisingly increased the valuation.

The implication of (i) is that the Shapley value ϕ i may be negative, which results in an unusable negative/undefined r ∗ i . However, this issue can be averted while preserving P3 by upweighting non-negative MCs, such as to the empty set, as mentioned in Footnote 10. The implication of (ii) is that some (large) noise realization can result in a more valuable model reward than the grand coalition's model, i.e., r i > v N . However, collaborating parties still prefer p ( θ | o N ) valued at v N as the more surprising model reward is not due to observations and information. This motivates us to define more specific desiderata (P1 and P2) for our reward scheme.

Lastly, one may question if we should change the valuation function. Should we use the information gain I ( θ ; o C ) = E o C [ v C ] on model parameters θ given perturbed SS o C instead to eliminate (i) and (ii)? No, the information gain is undesirable as it disregards the observed perturbed SS o C and will not capture a party's preference for higher similarity of its model reward to the grand coalition N 's posterior p ( θ | o N ) .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of learning a machine learning model collaboratively under differential privacy and the incentives for parties to participate in the effort. More specifically, in the papers setting, multiple parties are sharing private sufficient statistics to a central aggregator. DP induces noise to the sufficient statistics and parties with less strict privacy guarantees contribute more signal to the inference than the ones with strong privacy guarantees. Therefore, authors argue that there should be an incentive structure in participating in the scheme which rewards the parties with less strict privacy guarantees more. Besides accuracy, authors consider fairness as another desired property from the collaboration. Authors propose a method, that learns a posterior distribution for the parameter of interest using the shared private suff. stats., and releases tempered posterior samples to each client. The level of tempering will determine how close the samples are to the true posterior, and parties with stricter privacy guarantees get samples from more flattened (less tempered?) posterior. The level of tempering is selected by the central aggregator based on the Bayesian surprise which measure how much more information the parties data brings in (measure in terms of the KL divergence against the prior). Authors demonstrate empirically, that a party's reward from collaboration is a better model than simply using their own posterior learned from local data under DP. Furthermore the results show that the reward mechanism is able to give larger rewards for looser privacy guarantees. Finally, authors demonstrate that the proposed reward control mechanism outperforms an existing one (i.e. gets larger benefits from the collaboration), especially when the reward is low.\n\nSTRENGTHS:\nI believe the setting where multiple data holders would like to collaboratively learn from sensitive data is very realistic, and proposing a reasonable incentive structure for participation is a valuable goal. The proposed structure, that promises more reward for less perturbed data, sounds reasonable to me. The theoretical properties (especially V2 and V3) of KL as a valuation function are novel as far as I know, and give a solid theoretical foundation for using it. Using tempering to control the reward is also novel and sounds like a reasonable choice.\n\nThe empirical evaluation over multiple models and data sets gives support for the incentive structure, showing that a party can gain from the collaboration and as the $\\epsilon$ grows they are rewarded more.\n\nWEAKNESSES:\nThe method relies on a rather strong trust model that the parties contribute honestly to the collaboration. As the surprise, which determines the valuation for a party, is computed as the KL-divergence between the parties posterior and the prior, it would be quite easy for a party to send bogus data to the mediator that would yield a large KLD (especially if the prior is know). Authors do address this limitation in Remark in line 172 (and extensively in Appendix I), but I would encourage authors to extend this discussion in the main paper as well.\n\nQUESTIONS:\n- You say that you can optimize the $\\kappa_i$ to control $q_i(\\theta)$ using any root-finding algorithm. However, to solve the $q_i(\\theta)$ for a given $r_i^*$ you would need to optimize the KL-divergence between the tempered posterior and the prior right? As you say in Section 3, this KLD is intractable, and hence you need to deploy MCMC methods to evaluate it. How computationally expensive is this? Would you need to run the entire MCMC chain for each $\\kappa_i$ value from scratch? \n- Figures 2a and 2c: is the difference between $v_2$ and $v_N$ on the largest $\\epsilon$ due to some numerical issue in valuating the party? Or are they supposed to overlap if $r_2 = v_N$?\n- Minor comment: On line 238, you say that \"... party k will now enjoy $(1/\\tau)$-DP guarantee ...\". I guess you don't really mean this as pure DP guarantee, but as an RDP guarantee? If so, it would be good to add $\\lambda$ in the privacy notation (and call it RDP instead of DP).\n\nLIMITATIONS:\nThe limitations of the method are discussed in the main paper and more thoroughly in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis research paper investigates the intersection of data sharing incentives and privacy concerns within the realm of collaborative machine learning (ML). Collaborative ML aims to improve model quality by leveraging diversified data from multiple parties, yet the potential benefits of this practice are often hampered by concerns about privacy and the costs of sharing data.\nSeveral existing studies have acknowledged the need for incentives that encourage collaboration, such as guaranteed fair rewards for valuable data contributions. Yet, these incentive-based rewards expose the parties to privacy risks. Meanwhile, some solutions enforce differential privacy (DP) to mitigate privacy concerns but may inadvertently compromise the perceived fairness of collaboration and group welfare. This research fills the gap by proposing an incentive-aware, privacy-preserving reward scheme.\nThe authors address several questions in their investigation. They consider the impact on valuation and reward if a party demands stronger DP guarantee, suggesting that such a party's reward should generally decrease to avoid randomness due to DP noise. They propose a privacy-valuation trade-off and explore ways to value a party's data, focusing on the quality of inference of model parameters under DP. Finally, they detail a reward scheme designed to maintain privacy, individual rationality, and fairness.\n\nThe paper's significant contributions include the development of a new privacy-valuation trade-off criterion, the proposal of novel incentives, and the introduction of reward control mechanisms to adjust the distribution of posterior samples of model parameters among different parties. These solutions aim to preserve similarity to the grand coalition's model while deterring excessive DP.\n\nSTRENGTHS:\nA key strength of this paper is its addressing of a highly significant, real-world issue. Collaboration between diverse parties is a critical aspect of machine learning. However, progress in this field is often hindered by concerns surrounding privacy. Therefore, the paper's proposition of a framework that integrates privacy within the incentive scheme of a collaborative process is not just beneficial, but essential. It effectively merges theoretical constructs with pragmatic applications, fostering advancements in secure, collaborative machine learning practices. Moreover, the paper has good organization and clarity.\n\nWEAKNESSES:\nWhile the paper might appear to lack novelty as it adopts numerous techniques from previous work in collaborative machine learning, it does so within the context of differential privacy. This presents a unique perspective. The process of designing incentives that take into account privacy considerations necessitates an innovative approach.\n\nQUESTIONS:\nno questions\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a mechanism to do single-round private collaborative model learning by several agents, each with access to a dataset. The agents do not want to share their data and instead exchange perturbed sufficient statistics of their data, which the central server must aggregate and learn from. Since the devices benefit from collaboration, they might be incentivized to add excessive noise to their sufficient statistics to get the maximum privacy possible while still getting some benefit from collaboration. While this might benefit specific agents, it might not be the ideal scenario from the perspective of the majority of agents. This paper's mechanism disincentivizes this behavior by providing different models to each client, based on their differential privacy requirements, with better models to agents with \"most useful\" sufficient statistics. The rewards provided by the server also satisfy other desirable properties such as fairness, individual rationality, group welfare, etc.        \n\nOverall I like the paper. It is a good step towards reconciling federated learning with agents' strategic behavior. Apart from complaints about presenting the overall scheme, and other minor writing issues, I do not have major concerns about the paper and recommend accepting it. I would recommend the authors incorporate the suggestions below to improve the exposition. I am open to increasing my score.\n\nSTRENGTHS:\nThe paper addresses a significant problem: agents' strategic and self-serving behavior in federated learning. Federated learning hopes to enable large-scale privacy-preserving collaborations and encourage users to share the benefits of their data to develop an overall better model. Most incentive designs work with explicit monetary rewards, but without an actual server, that is not doable. This paper bypasses the issue and designs model rewards that satisfy several desirable properties. The paper is also quite exhaustive in its treatment of which desirable properties must be satisfied at each step.\n\nWEAKNESSES:\n1. The paper is written in a bottom-up approach, i.e., from sufficient statistics to the final model. It is OK to write it this way, but the overall scheme of things is unclear after reading the paper. I know figure one is an attempt to summarize everything, but an actual pseudocode would be helpful to show explicitly which quantities are being computed, how sampling occurs, etc. It would be helpful to also indicate in the pseudocode which sampling steps are approximated using MCMC procedures. For instance, currently, the way it's presented is unclear up to section four; why is the paper talking about coalitions? An informed reader might guess it is due to Shapley value computations, but otherwise, it seems a bit arbitrary. Writing the paper top-down or adding the above pseudocode might clarify these aspects. \n\n2. The individual rationality definition in P4 seems incorrect. Why would the agent perturb the sufficient statistic if they are alone? Collaboration with perturbed statistics should be better than unperturbed individual effort. \n\n3. Finally, the complexity of computation/sampling will also become explicit by being more precise about the exact computations. Shapley value computations are generally expensive, and discussing the effect of inexactness in the procedure would be good.\n\nQUESTIONS:\nSee comments above. Can actual individual rationality be satisfied by the mechanism?\n\nLIMITATIONS:\nThe paper should emphasize the scope of the work. Most federated learning happens over several interactions and involves local processing on the agent. There might be some applications where a single round of interaction between the server and the clients is enough, but that is an exception, not the norm. It is unclear if the mechanism provided in this paper can be applied repeatedly in some manner. I understand this extension might be complex, but not discussing it is incorrect. I urge the authors to list potential applications they have in mind in the motivation. As mentioned above, the authors should be more explicit about the exact computations and the complexity of running the mechanism. Finally, the individual rationality requirement in P4 seems wrong. If this can not be changed, it should be emphasized strongly that this is a limitation of the approach.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFrom a mostly empirical angle, the paper studies a new valuation metric for incentivizing agents to share their data for collaborative ML while ensuring the data they share is Renyi-DP. Particularly, the KL-divergence between agents' prior and posterior, or the Bayesian surprise, is used as the value of the (possibly perturbed) observations received by the agents. Finally, the proposed method is validated on a synthetic dataset, a regression dataset, and a classification dataset.\n\nSTRENGTHS:\n1. The experimental results are explained carefully, in detail, and the figures emphasize the key takeaways from the paper.\n2. Overall the paper is easy to follow and nicely presented.\n3. Ethical concerns and potential societal impact are discussed in depth, and in detail, in the appendix.\n4. The problem being studied is an interesting one and especially timely given current conversation on privacy and ethical concerns of large scale ML models.\n\nWEAKNESSES:\n1.  While the paper is mostly empirical and should be assessed accordingly, the proposed method still lacks mathematical justification. For instance, it is unclear for what choices of $\\rho$ (or if any) would IR be satisfied for all agents. Quantitative analysis of the desiderata for different values of $\\rho, \\lambda, \\epsilon$, especially quantifying the interplay between $\\rho$ and $(\\lambda, \\epsilon)$ Renyi-DP, would strengthen the results. At the current stage, it is unclear under what kind of problem settings or privacy constraints would the proposed mechanism be feasible.\n2.  In the same vein, desiderata **P5** and **P6** are only discussed intuitively. For ML practitioners with money at risk, additional discussions (and perhaps mathematical or game-theoretic justifications) would be more persuasive.\n3.  While Appendix I, Q5 has partially addressed the relationship between the work and [Sim et al., 2020], as discussed above, it is not clear how **P5** fit in the existing theoretical framework around Shapley values. Similarly, while the tempering technique has not been introduced in prior works, the mathematical or game-theoretic implications of $\\kappa$ are not addressed, and only an intuitive discussion is provided in the appendix. \n4.  The potential application of the method seem limited in domain. The method does not appear to be generalizable outside of Bayesian linear regression. Moreover, calculating the proposed values already entails a costly MCMC inference procedure in order to evaluate the Baeysian surprise. It is unclear if such procedures would remain practical even for smaller neural nets. (While the appendix has discussed various approximation schemes, it may be difficult to persuade practitioners to adopt the method without additional mathematical guarantees on how robust the proposed desiderata are to the estimation errors in both KL and posterior).\n5.  Desiderata **P2** is a strict relaxation of the weak efficiency constraint (R3) in [Sim et al., 2020]. Particularly it requires only one agent, as opposed to one agent in each group, to take full advantage of shared data, and may discourage agents from participating in the mechanism (especially with limited IR guarantees).\n\n- [Sim et al., 2020]. \"Collaborative machine learning with incentive-aware model rewards.\" International conference on machine learning. PMLR, 2020\n\nQUESTIONS:\n1.  Can the authors explain **P1** in more detail? From my understanding, it seems like **P1** means that agents&rsquo; reward cannot reveal too much information about other agents&rsquo; data. It would be great if the authors could confirm this. It might be me but currently the definition looks a bit confusing.\n\nLIMITATIONS:\nThe authors have discussed in detail the potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-23"}