Title: PI-FL: Personalized and Incentivized Federated Learning
Venue: NeurIPS 2023
Accepted: no
Keywords: Federated Learning

Personalized Federated Learning

Incentivized Federated Learning

Client Autonomy
Abstract: Existing incentive solutions for traditional Federated Learning (FL) only consider individual clients' contributions to a single global model. They are unsuitable for clustered personalization, where multiple cluster-level models can exist. Moreover, they focus solely on providing monetary incentives and fail to address the need for personalized FL, overlooking the importance of enhancing the personalized model's appeal to individual clients as a motivating factor for consistent participation. In this paper, we first propose to treat incentivization and personalization as interrelated challenges and solve them with an incentive mechanism that fosters personalized learning. Second, unlike existing approaches that rely on the aggregator to perform client clustering, we propose to involve clients by allowing them to provide incentive-driven preferences for joining clusters based on their data distributions. Our approach enhances the personalized and cluster-level model appeal for self-aware clients with high-quality data leading to their active and consistent participation. Through evaluation, we show that we achieve an 8-45% test accuracy improvement of the cluster models, 3-38% improvement in personalized model appeal, and 31-100% increase in the participation rate, compared to a wide range of FL modeling approaches, including those that tackle data heterogeneity and learn personalized models.

Abstract

Existing incentive solutions for traditional Federated Learning (FL) only consider individual clients' contributions to a single global model. They are unsuitable for clustered personalization, where multiple cluster-level models can exist. Moreover, they focus solely on providing monetary incentives and fail to address the need for personalized FL, overlooking the importance of enhancing the personalized model's appeal to individual clients as a motivating factor for consistent participation. In this paper, we first propose to treat incentivization and personalization as interrelated challenges and solve them with an incentive mechanism that fosters personalized learning. Second, unlike existing approaches that rely on the aggregator to perform client clustering, we propose to involve clients by allowing them to provide incentive-driven preferences for joining clusters based on their data distributions. Our approach enhances the personalized and cluster-level model appeal for self-aware clients with high-quality data leading to their active and consistent participation. Through evaluation, we show that we achieve an 8-45% test accuracy improvement of the cluster models, 3-38% improvement in personalized model appeal, and 31-100% increase in the participation rate, compared to a wide range of FL modeling approaches, including those that tackle data heterogeneity and learn personalized models.

1 Introduction

Training high-quality models using traditional distributed machine learning requires massive data transfer from the data sources to a central location, which raises various communication, computation, and privacy challenges. In response, Federated Learning (FL) [1-4] has emerged as a solution to train models at the source, reducing privacy issues and addressing the need for high-quality models. However, the success of FL relies on resolving various new challenges related to statistical heterogeneity [5-10], scheduling [11-13], and incentive distribution [14-18]. Recent works have focused on training personalized models [9,19-22] to overcome data heterogeneity challenges.

Among personalized Federated Learning (pFL) techniques, similarity-based approaches that use 27

28

clustering of clients at the aggregator have gained popularity [23-27]. These personalization solutions

fulfill the primary goal of overcoming data heterogeneity for specific cases. However, existing pFL 29

solutions do not include any incentive mechanism, which is crucial in FL to motivate participants 30

to contribute their data and computation resources. Existing incentive mechanisms [14,16,28] for 31

32

traditional FL cannot be applied to pFL techniques because they only consider the performance

contribution of clients towards training a single objective. In contrast, clients in pFL can be con33

tributing towards multiple objectives simultaneously [7, 8, 24, 25, 29, 30]. Furthermore, traditional 34

incentive solutions only provide monetary benefits and do not consider increasing personalized 35

models' appeal as an incentive for encouraging active and reliable participation of clients. Without 36

incentives, participants may provide low-quality data [14,16,18] or opt-out from participation 1 [31], 37 leading to poorly performing pFL models [10,32,33], as shown with empirical evaluations in the later 38 section. Collaboration fairness [34, 35] can also be ensured by appropriately rewarding contributions 39 and accounting for data heterogeneity [18,36]. 40

41

42

43

44

45

46

47

48

49

50

51

In addition, since existing pFL techniques assume voluntary and consistent participation from clients, the aggregator controls the client selection and training with limited knowledge of clients' training capacity, availability, frequency of new incoming data, clustering preferences, and performance requirements from the trained personalized models. These factors can directly influence the motivation of self-conscious clients to participate consistently. Our evaluation shows that this causes frequent opt-outs from uninterested clients due to uninformed clustering decisions by the server and low personalized model appeal (PMA) 2 , which leads to reduced pFL performance. We also show that solving personalization and incentivization as interrelated challenges yield better outcomes for pFL than solving them as separate problems. However, this requires new paradigms for clustered pFL using data distribution information available to clients via their preferences and designing incentive mechanisms for increasing pFL appeal to reduce client opt-outs.

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

In this paper, we propose PI-FL that combines clustering-based pFL with token-based incentivization. Unlike previous works that control clustering from the server side, PI-FL allows clients to estimate the importance of each cluster and send their preferences for joining them to the aggregator as bids. To identify a cluster's importance to a client we use the importance weight of the cluster model as defined by FedSoft [25]. Clients also use the importance weights to perform weighted local aggregation for single-shot personalization. This client-driven clustering approach results in accurate clustering because clients can attain a global perspective from their own local dataset which is only accessible to them and the importance weights information of each cluster. This allows them to make informed decisions that the server cannot make, resulting in improved PMA and reduced opt-outs. To incentivize clients for consistent participation, PI-FL motivates clients to join clusters with the clients that are most similar to them, maximizing their contribution to the cluster and, in turn, their rewards. Good quality cluster-level models then produce more appealing personalized models for each client. The incentive mechanism treats clients as both providers and consumers. As a consumer, the client tries to attain a certain level of personalized model appeal, so it pays the provider to spend resources to participate in training for the said model in each round. Whereas as a provider, the client earns a profit based on its marginal contribution to training the cluster models. The marginal contribution is calculated with a Shapley Value approximation due to the large computational overhead of the original algorithm [38-41].

Contributions. Existing pFL solutions fail to include PMA as an incentive to maintain consistent participation, resulting in increased opt-outs. To address this issue, we propose PI-FL as the first contribution, which provides contribution-based incentives to achieve collaborative fairness and maintain the cluster-level and personalized models' appeal for clients to prevent opt-outs. Additionally, PI-FL has the added advantage of creating personalized models for unseen clients with unknown data distributions that perform similarly to seen clients without the need for training. Secondly, we provide theoretical analysis and empirical verification of the benefits of including incentives with personalization. Lastly, we empirically evaluate the performance of PI-FL and other pFL models.

2 Related work

Cluster-based pFL: Among the cluster-based pFL works most related to PI-FL are FedSoft [25], FedGroup [24], and [29]. FedSoft utilizes soft clustering on the basis of matching data distributions in clients with cluster models while FedGroup quantifies the similarities between clients' gradients by calculating the Euclidean distance of decomposed cosine similarity metric and [29] finds the optimal personalization-generalization trade-off from the cluster model by solving a bi-level optimization problem. This work incurs clustering overhead at each iteration and does not consider the overlap of distribution between clients wherein each client is restricted to one cluster for each training round. Other cluster-based pFL models include IFCA [42] which proposes a framework for loss-based clustering of clients and [23] which proposes three approaches for personalization using clustering, data interpolation, and model interpolation.

Other pFL models: Some pFL models propose meta-learning techniques that provide methods for rapid training of a personalized model. These include fine-tuning methods such as Per-FedAvg [43]

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

and regularization of local models [44,45]. Others works [8,46] including FedALA [6], Ditto [30] and pFedMe [47] propose multi-task learning and model-interpolation [48] pFL models. FedFomo [7] suggests an adaptive local aggregation approach for personalization. FedProx [5] proposes a proximal term to improve the stability of FL. As per our knowledge, all of these pFL works lack qualities for attracting or sustaining long-term participation from self-conscious clients leading to an increase in opt-outs and low PMA. Moreover, most of these works require either require further training or re-clustering to adapt the personalized models for new incoming clients.

Incentivized FL: FAIR [14] integrates a quality-aware incentive mechanism with model aggregation to improve global model quality and encourage the participation of high-quality learning clients. FedFAIM [18] proposes a fairness-based incentive mechanism to prevent free-riding and reward fairness with Shapley value-based client contribution calculation. [31] proposes an approach based on reputation and reverse auction theory which selects and rewards participants by combining the reputation and bids of the participants under a limited budget. [16] proposes an approach where clients decide whether to participate based on their own utilities (reward minus cost) modeled as a minority game with incomplete information. Other incentivized FL works include [15,17,34,49,50]. All of these works propose standalone solutions to attract clients, however, none of them fulfill the design requirements to be used with any pFL models.

Why existing incentive mechanisms cannot be applied directly to pFL frameworks?

Existing FL incentivization schemes designed for motivating clients to contribute to a single global goal [14,16,18,31] may not be applicable to pFL frameworks due to the multi-dimensional goals and objectives involved. In pFL frameworks, multiple objectives must be optimized simultaneously, such as cluster and personalized models per client in cluster-based pFL [24, 25, 29] or global and local models per client in multi-task learning [20,30,45,47]. To encourage clients to contribute towards the multiple objectives in pFL frameworks, new incentive mechanisms need to be developed that are specifically tailored to their multi-objective nature. PI-FL uses clustering for pFL wherein the clusters memberships are changed after every R training rounds. PI-FL is different from these as it forms clear boundaries between multiple cluster models and improves shared learning between cluster similarities through multiple participation at the client level. PI-FL incorporates maintaining PMA for consistent client participation with an incentive mechanism that directly motivates personalized training on the basis of Individual Rationality (IR) constraint of game theory [14, 51].

3 Proposed Methodology

In this section, we introduce PI-FL, which has three main modules: the profiler, the token manager, and the scheduler as shown by the architecture diagram in Figure 1. The profiler calculates and maintains the history of client contributions using Shapley Values approximation (lines 24-27) of Algorithm 1. The profiler also aids the scheduler in forming clusters using two different modes further

Profiler

Scheduler

Token

Manager

Tier

Preferences

Create

tiers

Token bids

Pay Provider

Reimburse

Consumer

bids

Participation History

& Marginal Contributions

explained in section 3.1. The token manager orchestrates transactions, holds auctions, deducts payments, and distributes rewards as given in lines (13 and 14). The scheduler selects clients based on bids and contributions, grouping them for improved homogeneity shown in lines (20 and 27-29). Individual clients calculate the importance weights of each aggregated cluster model and send their preference bids to the Token Manager for joining clusters as shown in lines (23-28) in Algorithm 2. Clients also generate a single-shot personalized model, shown in line 29. We assume that each client will look to maximize their profits according to the principle of Individual Rationality (IR) [10, 52] and this will lead them to choose clusters in which they can contribute the most for maximum reward.

3.1 Profiler

At the start of pFL training, the scheduler module forms the initial clusters by randomly assigning clients. Then for each round, clients train the cluster-level model on their local data and calculate the importance weight of each aggregated cluster model M k on their local dataset via Equation 1. Here υ ck is the normalized sum of correctly predicted data points n ck on local dataset D c of client c . The importance weights are used to generate a single-shot personalized model through the weighted aggregation of cluster-level models using Equation 2.

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

Algorithm 1 PI-FL (Server)

Here the P ck is the personalized model of client c in cluster k and ω k is the weight vector of k cluster model. Using this, clients generate single-shot personalized models offline according to their dynamic data needs. The client-centric clustering and participation method enhances the appeal of pFL for clients and in doing so also provides them the opportunity to customize their personalized model offline in case their requirements which are unknown to the server change during training. Clients can also make informed decisions on participating in training clusters based on their budget and past rewards, using importance weights and knowledge of previous rounds. They convey their preferences to the aggregator by submitting bids for the cluster they wish to participate in for the next training round.

The profiler calculates the marginal contributions of each client after every round using Shapley Values approximation (Algorithm 3), aiding scheduling by providing data quality information to the scheduler. The Shapley Value approximation derivation from Appendix is used to avoid the computational expense of calculating Shapley Values for multiple clients. PI-FL also includes a mode to facilitate clients to form well-defined initial clusters. So the clients can avoid the decision-making process in the beginning and streamline their spending when the client contributions and cluster distributions are unclear. For this, the profiler and the scheduler module facilitate forming the initial clusters by training for some pre-training rounds. This is done as client contributions and similarity metrics that the clients use among other metrics to make decisions about joining clusters are initially unknown. After pre-training, the profiler calculates per-class F1-Scores ξ of all client local models on an IID test dataset [53]. Then the profiler with the help of scheduler clusters clients for the next training round using the K-Means clustering [55] algorithm with the most varying F1-scores V F 1 from C total classes. Equation 3 shows the calculation of V F 1 where C is the number of total classes and N is the number of all available clients.

Weperform all our evaluations for PI-FL without this feature, but this is an added feature that PI-FL in166 cludes for faster convergence and to save clients' costs. We also realize the constraints in choosing all 167 the clients for training, which is why clients that reply within threshold time in pre-training rounds are 168 used to calculate F1 scores. The remaining clients are considered unexplored and assigned to clusters 169 randomly, they can later settle into appropriate clusters through preference and contribution selection. 170

Algorithm 2 PI-FL (Client)

171

172

173

174

175

176

177

178

179

180

181

182

3.2 Token Manager

The token manager acts as a bank to orchestrate and keep track of transactions between different clients. At the start of each training round the token manager holds an auction for each cluster, and the clients that want to participate in that cluster place their bids using tokens. The token manager forwards the list of willing clients to the scheduler to select clients for training. It also deducts payments from the willing clients/consumers via Equation 4. Here τ i is the tokens owned by client i , ζ k are the clients willing to join cluster k , and τ p in this and all following Equations is the per round bid amount to be paid by each client for participation.

The tokens collected as payments from clients/consumers are then added to the available pool of tokens at the Token Manager as shown in Equation 5. Here τ ar are the total available pool of tokens at the Token Manager. The term N p is the number of clients selected on basis of performance and N r is the number of clients selected randomly. The significance of using N p and N r is explained in section 3.3.

The token manager handles the distribution of reimbursement and rewards to each provider/client. 183 Reimbursement penalizes degradation in the performance of providers and depends on the utility 184 function. The utility is calculated as the percentage of average accuracy improvement of the cluster 185 model M k over the maximum achieved accuracy in past rounds on the local data of clients in cluster 186 k . The utility function is given in Equation 6 and reimbursement calculation is given in Equation 187 7, both metrics are calculated at the profiler which assists the token manager in reimbursement. 188

189

In Equation 6, Acc kr is the cluster-level model accuracy in the current round r and Acc kmax is the 190 maximum cluster-level model accuracy achieved until the current round r . The term η represents the 191 maximum portion of tokens that can be returned and γ represents the maximum accuracy improvement 192 that leads to the use of one full token. In Equation 7, τ ar are the total number of tokens collected 193 from consumers/clients for r training round. We have used a similar approach to [28], however, they 194 use the accuracy of the FedAvg model on an IID dataset. It is not practical to assume the presence of 195 an IID dataset that can correspond to the data distribution of clients within a cluster which is why we 196 rely on the local dataset of clients within that cluster to gather this information. 197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

After reimbursement, the token manager uses the marginal contributions calculated by the profiler and sorts providers/clients by their contributions and participation record in Equation 8. Here ψ ki represents the marginal contributions and Ω ki represents the participation records of all clients N in K clusters. The term β is a normalizing term from Equation 8 in which N r are the number of providers selected for participation in round r . Using the ranks α of providers from sorting and the normalization term β , the remaining available tokens are distributed between these providers in Equation 8. Here τ i represents the tokens owned by provider/client i and τ ar are the tokens available for incentive distribution at the token manager. Through reimbursements to consumers and payments to providers, the Token Manager ensures that each client receives an incentive according to their contributions in training the pFL models. By doing so, PI-FL incentivizes improvement in personalized learning, resulting in an enhancement of PMA and a decrease in opt-outs.

3.3 Scheduler

The scheduler selects clients for each round r by the SelectClients ( r ) function given in Algorithm 1. The scheduler receives the preference bids θ i from the token manager, the marginal contributions ψ ki from the profiler for each client i ∈ N in cluster k ∈ K , where N is the total number of clients and K are the total number of clusters. Using this information scheduler groups clients with similar preference bids and then sorts those clients by their marginal contributions. Then the scheduler selects N p number of clients from the sorted clients and N r number of clients randomly. Both N p and N r are tunable parameters. To reduce bias, a small portion of clients N r are selected randomly which is a technique adopted from previous works [2,28,56,57]. By grouping clients with similar preferences the scheduler reduces the within-cluster bias improving the within-cluster homogeneity and a cluster model is produced that accurately represents the clients within it. Section 4 gives a theoretical analysis of how this is an important factor in improving the PMA.

4 Theoretical Analysis

We study the following particular case to develop insights. Suppose there are m clients in total, each observing a set of independent Gaussian observations z i,j ∼ N ( µ i , σ 2 ) , j = 1 , . . . , n i , with a personalized task of estimating its unknown mean µ ∈ R . The quality of the learning result, denoted by ˆ µ , will be assessed by the mean squared error E i (ˆ µ -µ ) 2 , where the expectation E i is taken with respect to the distribution of client i .

It is conceivable that if clients' underlying parameters µ i 's are arbitrarily given, personalized FL may not boost the local learning result. To highlight the potential benefit of cluster-based modeling, we suppose that the m clients can be partitioned into two subsets: one with m 1 clients, say T 1 = { 1 , . . . , m 1 } , and the other with m 2 clients, say T 2 = { m 1 +1 , . . . , m } , whose underlying parameters are randomly generated in the following way:

Here, β 1 and β 2 can be treated as the root cause of two underlying clusters. We will study how the values of sample size n i , data variation σ , within-cluster similarity as quantified by τ , and cross-cluster similarity as quantified by | β 1 -β 2 | will influence the gain of a client in personalized learning. To simplify the discussion, we will assess the learning quality (based on the mean squared error) of any particular client i in the following three procedures:

Local training : Client i only performs local learning by minimizing the local loss L i ( µ ) = ∑ n i j =1 ( µ -z i,j ) 2 , and obtains ˆ µ i = n -1 i ∑ n i j =1 z i,j . Thus, the corresponding error is

̸

Federated training : Suppose the FL converges to the global minimum of the loss, ∑ m i =1 n i n L i ( µ ) , n ∆ = ∑ m i =1 n i , which can be calculated to be ˆ µ FL = ∑ m i =1 n i n ˆ µ i . Consider any particular client i . Without loss of generality, suppose it belongs to cluster 1, namely i ∈ T 1 . From the client i 's angle, conditional on its local µ i and assuming a flat prior on β 1 and β 2 , client j 's µ j follows µ j | µ i ∼ N ( µ 1 , 2 τ 2 ) for j ∈ T 1 and j = i , and µ j | µ i ∼ N ( µ 1 + β 2 -β 1 , 2 τ 2 ) for j ∈ T 2 . Then, the corresponding error is

̸

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

It can be seen that compared with (10), the above FL error can be non-vanishing if ∑ j ∈ T 2 n j n ( β 2 -β 1 ) is away from zero, even if sample sizes go to infinity. In other words, in the presence of a significant difference between the two clusters, the FL may not bring additional gain compared with local learning.

Cluster-based personalized FL : Suppose our algorithm allows both clusters to be correctly identified upon convergence. Consider any particular client i . Suppose it belongs to Cluster 1 and will use a weighted average of Cluster-specific models. Specifically, the Cluster 1 model will be the minimum of the loss ∑ j ∈ T 1 n j n T1 L j ( µ ) , n T1 ∆ = ∑ j ∈ T 1 n j , which can be calculated to be ˆ µ T1 = ∑ j ∈ T 1 n j n T1 ˆ µ i . By a similar argument as in the derivation of (11), we can calculate

̸

The above value can be smaller than that in (10). To see this, let us suppose the sample sizes n i 's are all equal to, say n 0 , for simplicity. Then, we have

which is smaller than (10) if and only if

We derive the following intuitions from this analysis: R1. If the within-cluster bias is relatively small, the number of cluster-specific clients is large, and data noise is large, a client will have personalized gain from collaborating with others in the same cluster. R2. PI-FL's incentive algorithm rewards accuracy improvement reflected in PMA, which directly correlates with reducing within-cluster bias as per Equation 13. R3. By association, the incentive algorithm motivates clients to join similar clusters which increases cluster homogeneity and reduces the within-cluster bias. We show the impact of change in performance with an ablation study of PI-FL incentive in section 5.5.

5 Experimental Study

5.1 Experimental Setup

We use NVIDIA GeForce RTX 3070 GPUs for all our experiments. To evaluate the performance of PI-FL with other pFL models we use four datasets. A simple CNN model (32x64x64 convolutional and 3136x128 linear layer parameters) is used that can be trained on client devices with limited system resources to map Cross-Device FL settings [10] for all pFL methods.

CIFAR10 Data. For comparison with FedSoft [25] we use the same CIFAR10 dataset provided in their repository. This image dataset has images of dimension 32 × 32 × 3 and 10 output classes. We copy different data heterogeneity conditions from [25], namely 10:90, 30:70, linear, and random. The data classes are divided into two clusters D A and D B . In the 10:90 partition, 50 clients have 90% training data from D A and 10% from D B , while the other 50 have 10% training data from D A and 90% from D B . The 30:70 partition is similar to 10:90 except that the distribution ratios are 30% and 70% .

EMNIST Data. This image dataset has images of dimension 28 x 28 and 52 output classes where 26 classes are lower case letters and 26 classes are upper case letters. Same as CIFAR10 data, we use the 10:90 and 30:70 data partitions and also include linear, and random partitions. In linear partition, client k has (0 . 5 + k )% training and testing data from D A and (99 . 5 -k )% training data and testing data from D B . In the random partition, client k is assigned a mixture vector generated randomly by dividing the [0 , 1] range into S segments with S -1 points drawn from Uniform (0 , 1) . The training and testing data are then assigned based on this vector from D A and D B . Similar to [6, 30, 37], we also divide the EMNIST dataset into K clusters, where K = C t C p , C t are total classes and C p are the classes owned per party with no overlap of data between clusters.

Synthetic CIFAR10. This is a synthetic dataset created from the CIFAR10 dataset and contains the same hetrogenous partitions of 10:90, 30:70, linear, and random. The only difference is that

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

the training and testing data distributions are different to simulate dynamic data at the clients. For example, in 10:90 50 clients have 90% training data with 10% testing data from D A and 10% training data with 90% testing data from D B and vice versa. Similar to this, all the other partitions also have inverse training and testing data distributions. The reason for separate training and testing data distributions are explained in further depth in Appendix.

5.2 Focus of Experimental Study

First, we compare the clustering ability of PI-FL with a recent clustering-based pFL algorithm [25]. Second, we show how PI-FL compares with other non-clustering pFL models with a simple test accuracy comparison. Taking it one step further, we provide a comparison of PI-FL and other clustering and non-clustering pFL models in terms of reduction in opt-outs and PMA maintenance in section 5.3. Lastly, in section 5.5 we show that including client preferences while clustering yields better personalization results because clients can make decisions based on knowledge restricted to the aggregator server.

5.3 Test Accuracy performance study.

Effectiveness of clustering. We evaluate the performance of cluster-level models using holdout datasets sampled from the corresponding cluster distributions ( D A and D B ). To demonstrate the effectiveness of our proposed PI-FL method, we compare it with a recent cluster-based pFL algorithm called FedSoft using CIFAR10 data. We use the same parameters as in [25], with N = 100 clients, batch size 128, and learning rate η = 0 . 01 , and perform training for 300 rounds. Table 1 presents the test accuracy for the 10:90 and 30:70 partitions with PI-FL. We observe that PI-FL performs better for the 10:90 partition, where each cluster dominates one of the distributions. With PI-FL, clients that have a greater portion of data from θ 0 prefer to train in cluster c 0 , achieving 63 . 68% accuracy, while clients with a greater portion of data from θ 1 prefer to train in cluster c 1 , achieving 63 . 82% accuracy. FedSoft cluster-level models, on the other hand, achieve 50 . 7% and 49 . 6% for 10:90 data. It is worth noting that FedSoft is unable to cater to different partitions of data through its clustering mechanism, and the performance is adversely impacted by increased heterogeneity. Moreover, cluster-level models in FedSoft are unable to dominate a single distribution of data. As expected, the performance for the 30:70 partition is not as good as it is a less heterogeneous partition than the 10:90 partition. Neither cluster dominates a single distribution, and the clients with different distributions are not clearly differentiated for training with different clusters. Additionally, the cluster-level models c 0 and c 1 have similar performance with either distribution ( θ 0 and θ 1 ), as FedSoft promotes personalizing models when clients have a greater percentage of shared data. This generates cluster-level models that cannot represent a single distribution and do not perform as well as PI-FL with non-IID data.

Comparison with non-clustering pFL models. Table 2 shows a test accuracy comparison of PI-FL with other recent pFL algorithms. This table shows that some pFL models are able to perform well for individual partitions such as Ditto for 10:90, FedProx and FedALA for Linear, and PerFedAvg for Random, however, PI-FL is able to maintain its performance for all partitions.

5.4 Effectiveness of PI-FL in opt-outs reduction and PMA maintenance.

Each client's natural aim is to create a model that maximizes its test accuracy. Clients can have different thresholds of how much should be the least accuracy gain for it to participate in pFL, and we define this self-defined threshold as ρ i , i ∈ [ N ] . Since each client can have its own definition of the threshold requirement, we define ρ i as the test accuracy achieved by client i if it used FedAvg. So PMA i shows the gain in performance from pFL compared to vanilla FL using FedAvg for client i in N . PMA is similar to GMA from [37], however, creating a single global model may not be appealing for all clients as we show in section 4 and verify in section 5.4. We formally define PMA and opt-outs in Equation 14 and 15 respectively, where f i ( w k ) is the test accuracy achieved by pFL.

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

20

0

20

40

PMA (%)

(a) PI-FL & FedSoft

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

40

20

0

(b) PI-FL & Ditto

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

40

20

0

20

(c) PI-FL & FedALA

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

40

20

0

20

(d) PI-FL & FedFomo

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

40

20

0

20

PMA (%)

(e) PI-FL & FedProto

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

40

20

0

20

(f) PI-FL & FedAMP

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

0

20

40

PMA (%)

(g) 4 classes per client

Figure 2 shows the empirical Cumulative Distribution Function (CDF) plot of PMA for all clients with CIFAR10 data using FedSoft and PIFL and with EMNIST dataset for all other pFL models. PI-FL particularly outperforms for the 10:90 partition in terms of PMA as this is the most heterogeneous data partition as can be seen in Figure 2a. The EMNIST dataset is less heterogeneous as it has more classes per client compared to CIFAR10 which is why FedAvg is able to perform relatively well and there is less room for improvement with personalizing. PI-FL maintains the PMA and also improves it, particularly for the 10:90 and 30:70 partitions where other pFL solutions lack. We also test on a more heterogenous case where the dataset is divided into 52 clusters and each client owns 4 maximum classes. Figure 2g shows that while other pFL solutions perform better than FedAvg only Ditto and FedProto come relatively close to PI-FL, however, PI-FL outperforms them both by approximately 15% in terms of PMA. The FedProx, FedALA, and PerFedAvg opt-out ratios are 0.64, 0.31, and 0.68, respectively. Ditto, FedFomo, and PI-FL have no opt-outs. This goes to show that PI-FL is not only able to reduce the opt-outs but also improves the PMA under all data heterogeneity conditions.

5.5 Advantages of including client preferences in pFL.

We show that PI-FL can maintain the test accuracy of personalized models even in case of dynamic data at the client or a new unseen client accidentally being added to the wrong cluster. Figure 3 shows the CDF of clients' personalized model test accuracy after training for 500 rounds. PI-FL is robust to variations in clients' local data, while FedSoft is less effective due to its clustering approach being based on the server's perspective, which lacks access to clients' private data and limits its ability to make accurate clustering decisions.

Ablatian study with Incentive in PI-FL. To measure the impact of incentive provision on personalized model generation we test PI-FL with incentives enabled and disabled. Figure 4 shows the CDF of clients' personalized model test accuracy with the Synthetic CIFAR10 dataset. Except for the 30:70 partition, the accuracy for all other partitions is higher with the incentive enabled. We argue that the test accuracy for 30:70 is low in this case because it is a less heterogeneous data case and PI-FL performs best in cases where data is highly heterogeneous and requires personalized learning. Further details of the experimental setup and impact of incentive on clustering are discussed in the Appendix.

6 Conclusion

In this paper, we proposed PI-FL to address the challenges of incentive provision in pFL for increasing consistent participation by providing appealing personalized models to clients. PI-FL client-centric clustering approach ensures accurate clustering and improved performance even in case of dynamic data distribution shift of the client's local data or inadvertently mistaken clustering decision by the client. Unlike prior works that consider incentivizing and personalization as separate problems, PI-FL solves them as interrelated challenges yielding improvement in pFL performance. Extensive empirical evaluation shows its promising performance compared to other state-of-the-art works.

0

20

40

60

80

Accuracy (%)

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

PI-FL (10:90)

FedSoft (10:90)

PI-FL (30:70)

FedSoft (30:70)

PI-FL (linear)

FedSoft (linear)

PI-FL (random)

FedSoft (random)

0

20

40

60

80

Accuracy (%)

0.0

0.2

0.4

0.6

0.8

1.0

Number of clients

10:90 (I)

10:90 (NI)

30:70 (I)

30:70 (NI)

Random (I)

Random (NI)

Linear (I)

Linear (NI)

375

References

J. Koneˇ cn` y, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, 'Federated 376 learning: Strategies for improving communication efficiency,' arXiv preprint arXiv:1610.05492 , 377 2016. 378

H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, 'Federated learning of deep 379 networks using model averaging,' CoRR , vol. abs/1602.05629, 2016. [Online]. Available: 380 http://arxiv.org/abs/1602.05629 381

382

383

384

E. Diao, J. Ding, and V . Tarokh, 'HeteroFL: Computation and communication efficient federated learning for heterogeneous clients,' in International Conference on Learning Representations , 2021.

385

386

387

388

--, 'Semifl: Semi-supervised federated learning for unlabeled clients with alternate training,' Advances in Neural Information Processing Systems , vol. 35, pp. 17 871-17 884, 2022.

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, 'Federated optimization in heterogeneous networks,' 2020.

J. Zhang, Y. Hua, H. Wang, T. Song, Z. Xue, R. Ma, and H. Guan, 'Fedala: Adaptive local 389 aggregation for personalized federated learning,' 12 2022. 390

M. Zhang, K. Sapra, S. Fidler, S. Yeung, and J. M. Alvarez, 'Personalized federated learning 391 with first order model optimization,' 2021. 392

O. MARFOQ, G. Neglia, A. Bellet, L. Kameni, and R. Vidal, 'Federated multi-task learning 393 under a mixture of distributions,' in Advances in Neural Information Processing Systems , 394 A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., 2021. [Online]. Available: 395 https://openreview.net/forum?id=YCqx6zhEzRp 396

397

398

399

V. Kulkarni, M. Kulkarni, and A. Pant, 'Survey of personalization techniques for federated learning,' in 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4) , 2020, pp. 794-797.

P. Kairouz, H. B. McMahan, A. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, 400 C. Charles, G. Cormode, R. Cummings et al. , 'Advances and open problems in federated 401 learning,' Foundations and Trends in Machine Learning , vol. 12, no. 3-4, pp. 1-357, 2019. 402

F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury, 'Oort: Efficient federated learning via 403 guided participant selection,' 2021. 404

405

406

407

J. Han, A. F. Khan, S. Zawad, A. Anwar, N. B. Angel, Y. Zhou, F. Yan, and A. R. Butt, 'Heterogeneity-aware adaptive federated learning scheduling,' in 2022 IEEE International Conference on Big Data (Big Data) , 2022, pp. 911-920.

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

Z. Chai, A. Ali, S. Zawad, S. Truex, A. Anwar, N. Baracaldo, Y. Zhou, H. Ludwig, F. Yan, and Y. Cheng, 'Tifl: A tier-based federated learning system,' CoRR , vol. abs/2001.09249, 2020. [Online]. Available: https://arxiv.org/abs/2001.09249

Y. Deng, F. Lyu, J. Ren, Y.-C. Chen, P. Yang, Y. Zhou, and Y. Zhang, 'Fair: Quality-aware federated learning with precise user incentive and model aggregation,' in IEEE INFOCOM 2021 - IEEE Conference on Computer Communications , 2021, pp. 1-10.

M. Tang and V. W. Wong, 'An incentive mechanism for cross-silo federated learning: A public goods perspective,' in IEEE INFOCOM 2021 - IEEE Conference on Computer Communications , 2021, pp. 1-10.

M. Hu, D. Wu, Y. Zhou, X. Chen, and M. Chen, 'Incentive-aware autonomous client participation in federated learning,' IEEE Transactions on Parallel and Distributed Systems , vol. 33, no. 10, pp. 2612-2627, 2022.

J. S. Ng, W. Y. B. Lim, Z. Xiong, X. Cao, D. Niyato, C. Leung, and D. I. Kim, 'A hierarchical incentive design toward motivating participation in coded federated learning,' IEEE Journal on Selected Areas in Communications , vol. 40, no. 1, pp. 359-375, 2022.

Z. Shi, L. Zhang, Z. Yao, L. Lyu, C. Chen, L. Wang, J. Wang, and X.-Y. Li, 'Fedfaim: A model 423 performance-based fair incentive mechanism for federated learning,' IEEE Transactions on Big 424 Data , pp. 1-13, 2022. 425

A. Z. Tan, H. Yu, L. Cui, and Q. Yang, 'Towards personalized federated learning,' CoRR , vol. 426 abs/2103.00710, 2021. [Online]. Available: https://arxiv.org/abs/2103.00710 427

428

429

430

H. Chen, J. Ding, E. W. Tramel, S. Wu, A. K. Sahu, S. Avestimehr, and T. Zhang, 'Self-aware personalized federated learning,' Advances in Neural Information Processing Systems , vol. 35, pp. 20 675-20 688, 2022.

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

L. Collins, E. Diao, T. Roosta, J. Ding, and T. Zhang, 'Perfedsi: A framework for personalized federated learning with side information,' 2022.

Q. Le, E. Diao, X. Wang, A. Anwar, V. Tarokh, and J. Ding, 'Personalized federated recommender systems with private and partially federated autoencoders,' arXiv preprint arXiv:2212.08779 , 2022.

Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh, 'Three approaches for personalization with applications to federated learning,' ArXiv , vol. abs/2002.10619, 2020.

M. Duan, D. Liu, X. Ji, R. Liu, L. Liang, X. Chen, and Y. Tan, 'Fedgroup: Accurate federated learning via decomposed similarity-based clustering,' 2021.

Y. Ruan and C. Joe-Wong, 'Fedsoft: Soft clustered federated learning with proximal local updating,' in AAAI , 2022.

X. Tang, S. Guo, and J. Guo, 'Personalized federated learning with contextualized generalization,' in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22 , L. D. Raedt, Ed. International Joint Conferences on Artificial Intelligence Organization, 7 2022, pp. 2241-2247, main Track. [Online]. Available: https://doi.org/10.24963/ijcai.2022/311

C. Ye, R. Ghanadan, and J. Ding, 'Meta clustering for collaborative learning,' Journal of Computational and Graphical Statistics , pp. 1-10, 2022.

J. Han, A. F. Khan, S. Zawad, A. Anwar, N. B. Angel, Y. Zhou, F. Yan, and A. R. Butt, 'Tiff: Tokenized incentive for federated learning,' in 2022 IEEE 15th International Conference on Cloud Computing (CLOUD) , 2022, pp. 407-416.

X. Tang, S. Guo, and J. Guo, 'Personalized federated learning with clustered generalization,' ArXiv , vol. abs/2106.13044, 2021.

T. Li, S. Hu, A. Beirami, and V . Smith, 'Federated multi-task learning for competing constraints,' CoRR , vol. abs/2012.04221, 2020. [Online]. Available: https://arxiv.org/abs/2012.04221

J. Zhang, Y. Wu, and R. Pan, 'Incentive mechanism for horizontal federated learning based on reputation and reverse auction,' in Proceedings of the Web Conference 2021 , ser. WWW '21. New York, NY, USA: Association for Computing Machinery, 2021, p. 947-956. [Online]. Available: https://doi.org/10.1145/3442381.3449888

T. Jahani-Nezhad, M. A. Maddah-Ali, S. Li, and G. Caire, 'Swiftagg: Communication-efficient and dropout-resistant secure aggregation for federated learning with worst-case security guarantees,' in 2022 IEEE International Symposium on Information Theory (ISIT) , 2022, pp. 103-108.

J. So, B. Güler, and A. S. Avestimehr, 'Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning,' IEEE Journal on Selected Areas in Information Theory , vol. 2, no. 1, pp. 479-489, 2021.

L. Gao, L. Li, Y. Chen, W. Zheng, C. Xu, and M. Xu, 'Fifl: A fair incentive mechanism for federated learning,' in Proceedings of the 50th International Conference on Parallel Processing , ser. ICPP '21. New York, NY, USA: Association for Computing Machinery, 2021. [Online]. Available: https://doi.org/10.1145/3472456.3472469

Y. Shi, H. Yu, and C. Leung, 'Towards fairness-aware federated learning,' IEEE Transactions on Neural Networks and Learning Systems , pp. 1-17, 2023.

Z. Zhou, L. Chu, C. Liu, L. Wang, J. Pei, and Y. Zhang, 'Towards fair federated learning,' in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery Data Mining , ser. KDD '21. New York, NY, USA: Association for Computing Machinery, 2021, p. 4100-4101. [Online]. Available: https://doi.org/10.1145/3447548.3470814

Y. J. Cho, D. Jhunjhunwala, T. Li, V. Smith, and G. Joshi, 'Maximizing global model appeal in federated learning,' 2023.

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

S. Wei, Y. Tong, Z. Zhou, and T. Song, Efficient and Fair Data Valuation for Horizontal Federated Learning , 11 2020, pp. 139-152.

A. Heuillet, F. Couthouis, and N. Díaz-Rodríguez, 'Collective explainable ai: Explaining cooperative strategies and agent contribution in multiagent reinforcement learning with shapley values,' IEEE Computational Intelligence Magazine , vol. 17, no. 1, pp. 59-71, 2022.

Z. Liu, Y. Chen, H. Yu, Y. Liu, and L. Cui, 'Gtg-shapley: Efficient and accurate participant contribution evaluation in federated learning,' ACM Trans. Intell. Syst. Technol. , vol. 13, no. 4, may 2022. [Online]. Available: https://doi.org/10.1145/3501811

L. Dong, Z. Liu, K. Zhang, A. Yassine, and M. S. Hossain, 'Affordable federated edge learning framework via efficient shapley value estimation,' Future Generation Computer Systems , 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X23001826

A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, 'An efficient framework for clustered federated learning,' Advances in Neural Information Processing Systems , vol. 33, pp. 19 58619 597, 2020.

A. Fallah, A. Mokhtari, and A. Ozdaglar, 'Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach,' in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 3557-3568. [Online]. Available: https://proceedings.neurips. cc/paper_files/paper/2020/file/24389bfe4fe2eba8bf9aa9203a44cdad-Paper.pdf

F. Hanzely and P. Richtárik, 'Federated learning of a mixture of global and local models,' 2021.

F. Hanzely, B. Zhao, and M. Kolar, 'Personalized federated learning: A unified framework and universal optimization techniques,' ArXiv , vol. abs/2102.09743, 2021.

V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, 'Federated multi-task learning,' in Advances in Neural Information Processing Systems , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/file/ 6211080fa89981f66b1a0c9d55c61d0f-Paper.pdf

C. T. Dinh, N. H. Tran, and T. D. Nguyen, 'Personalized federated learning with moreau envelopes,' 2022.

P. P. Liang, T. Liu, L. Ziyin, N. B. Allen, R. P. Auerbach, D. Brent, R. Salakhutdinov, and L.-P. Morency, 'Think locally, act globally: Federated learning with local and global representations,' 2020.

R. Zeng, S. Zhang, J. Wang, and X. Chu, 'FMore: An incentive scheme of multi-dimensional auction for federated learning in MEC,' in 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS) . IEEE, nov 2020. [Online]. Available: https://doi.org/10.1109%2Ficdcs47774.2020.00094

P. Sun, H. Che, Z. Wang, Y. Wang, T. Wang, L. Wu, and H. Shao, 'Pain-fl: Personalized privacypreserving incentive for federated learning,' IEEE Journal on Selected Areas in Communications , vol. 39, no. 12, pp. 3805-3820, 2021.

L. Zhang, T. Zhu, P. Xiong, W. Zhou, and P. S. Yu, 'A robust game-theoretical federated learning framework with joint differential privacy,' IEEE Transactions on Knowledge and Data Engineering , vol. 35, no. 4, pp. 3333-3346, 2023.

L. Li, Q. Li, H. Chen, and Y. Chen, 'Federated learning with strategic participants: A game theoretic approach,' in Proceedings of the 37th International Conference on Machine Learning . PMLR, 2020, pp. 8597-8606.

F1-scores, 'Sklearn.metrics.f1_score.' [Online]. Available: https://scikit-learn.org/stable/ modules/generated/sklearn.metrics.f1_score.html

Python, 'Cpython/functions.rst at main · python/cpython.' [Online]. Available: https: //github.com/python/cpython/blob/main/Doc/library/functions.rst

K-Means, 'Sklearn.cluster.kmeans.' [Online]. Available: https://scikit-learn.org/stable/modules/ generated/sklearn.cluster.KMeans.html

K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, 529 J. Koneˇ cný, S. Mazzocchi, H. B. McMahan, T. Van Overveldt, D. Petrou, D. Ramage, and 530 J. Roselander, 'Towards federated learning at scale: System design,' 2019. [Online]. Available: 531 https://arxiv.org/abs/1902.01046 532

A. Khan, Y. Li, A. Anwar, Y. Cheng, T. Hoang, N. Baracaldo, and A. Butt, 'A distributed and 533 elastic aggregation service for scalable federated learning systems,' 2022. [Online]. Available: 534 https://arxiv.org/abs/2204.07767 535

A. E. Roth, 'Introduction to the shapley value,' The Shapley value , pp. 1-27, 1988. 536

G. D. P. Regulation, 'General data protection regulation (gdpr),' Intersoft Consulting, Accessed 537 in October , vol. 24, no. 1, 2018. 538

A. Act, 'Health insurance portability and accountability act of 1996,' Public law , vol. 104, p. 539 191, 1996. 540

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes and analyzes a mechanism for incentivizing data contributions to federated learning systems in a context where contributors consider both monetary incentives for data contributions and the ability to experience the benefits of personalized systems.\n\nThe paper provides background on incentivized federated learning, describing the problem setting and proposed mechanisms, and theoretically analyzes a particular subset of this problem category to understand impacts of how underlying data that’s being modeled is distributed. Then, the authors perform experiments with classification datasets, with the assumption that classification datasets that might be under the control of a single entity in a classical setting are distributed between 100 clients and all data points belong to one of two clusters.\n\nSTRENGTHS:\nThe paper has a number of strengths:\n\nThe paper begins with a very helpful overview of work in federated learning. The paper will be of interest beyond the federated learning-focused sub-community. The current draft provides strong motivation by explaining the large gap in incentivized federated learning. It seems with further development this approach could be impactful in practice.\n\nThe paper combines theoretical analysis of a new mechanism (which has the key insight of accounting for both monetary incentives and personalization-as-an-incentive) with learning experiments. This combination can help convince readers the proposed approach is useful. While there is some room to provide more details about the intended \"data contributor scenario\" and \"data user scenario\" (see below), overall I think the methods lean more towards being an overall strength vs. a weakness.\n\nPresentation of results was strong overall.\n\nWEAKNESSES:\nWhile readers will appreciate the combination of theoretical analysis and experiment, it was not entirely clear exactly how the specific research questions being answered in each section were chosen (i.e., the “intuitions” R1-R3 seem useful but were not well prefaced). The experiments could be better motivated (more below in \"Questions\").\n\nThe paper is pretty abstracted away from any specific platform or use case where data contributors and data users interface via a particular FL architecture.\n\nThere may be an opportunity to strengthen the paper by specifying which ML use cases / contexts map well to the needed assumptions. In particular, while the paper relies on the existence of prior work on FL to motivate the need for FL, it’s not clear how many platforms running actual incentive systems (e.g. data markets, markets for crowdwork, etc.) are also supporting FL.\n\nIn general, I worry this paper may undersell its contribution by not providing enough details about what an on-the-ground implementation of the system would look like. Of course, this same criticism can be applied to a large body of similar work (including the RW here) and so it's not existential. Given the specific intended contribution here of proposing a new way to think about incentives in PFL, emphasizing plausibility may be especially valuable.\n\nQUESTIONS:\n-\tA core motivating claim of the paper is that ML requires an explicit incentive system for data collection. I personally think this a good aspirational goal for the community, but was skeptical when reading that this is an accurate claim, given major research attention being paid to models that primarily focus on scraped data (that doesn’t really map to traditional notions of consent or incentives). This doesn’t invalidate any claims here, but I think is important to discuss and acknowledge. For instance, see the claims in line 30 of the current draft.\n-\tOutside of the experiments, the current draft is not specific about any particular use case or specific kind of data generation scenario. For instance, in the paragraph starting on line 50, I had a decent sense of the abstract ideas being presented but had a lot of trouble imagine the specific scenarios faced by data contributing users. What specific kind of data are they imagining being incentivized to share? To me, this feels like a very valuable question to at least partially address given the core contribution of the paper is about incentives facing people.\n-\tThere are opportunities to further discuss the gap between modeling personalization in terms of traditional supervise classification with clusters versus something like a recommender system. As a reader, I was hoping to see some mention of this, as the conceptual contribution of the paper that personalization should be consider part of the incentive mechanism seems especially relevant to actual recommendation tasks.\n\nLIMITATIONS:\nThere are opportunities for the paper to be a bit more upfront about the limitations of focusing on incentive systems without discussing any specific populations of data contributors or specific use cases (beyond computer vision evaluation). While this paper seems to be part of a longer conversation between research papers that is primarily mathematical and leans heavily on assumptions about rational agents with programmatic approaches to computing utility, given that a key goal of having practical advantages mentioned in Related Work, addressing some practical use cases could make the work more convincing.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed PI-FL, an incentive-aware federated learning method that produces clustered personalization of client models. This work uniquely considers the client contribution assessment in the multi-objective setting of clustered personalization. The clients are also given the freedom to choose the cluster based on their incentives. Therefore, this paper could be valuable in exploring the topic of incentivization in the very specialized area of clustered personalization. Rather extensive empirical experiments were also carried out by the authors to demonstrate the practicability and effectiveness of PI-FL.\n\nSTRENGTHS:\n1. Recognition that personalization and incentivization in FL should be considered as interrelated challenges. In fact, incentivization is easier to be achieved when client models are allowed to be personalized.\n2. In addition to the proposed method, the paper provides good theoretical analyses of it through toy examples that are easy to understand. The analysis also offers good and interpretable insights.\n3. The types of empirical experiments carried out are rather extensive and demonstrate the effectiveness of the method. However, the comparison to baselines and the dataset choices can be further improved to be more comprehensive (sometimes omitting certain settings for certain methods).\n\nWEAKNESSES:\n1. This paper might need more concise and precise writing in Section 3 Proposed Methodology. The description of the methodology is long-winded and many design choices are not backed by theoretical justifications.\n2. The theoretical analysis focuses on the gain from collaboration but neglects the theoretical aspects of the mechanism that incentivizes participation and honest data contribution.\n\nQUESTIONS:\n**Major**\n\n1. The role of tokens in the incentive mechanism is unclear in the writing. Do all the clients start with the same number of tokens? Do they pay for the tokens after use? If tokens are involved in the incentivization, I do not see descriptions of how they interplay with model rewards (e.g., PMA). For example, should we ask for more money from an agent that achieves more PMA? Is it still fair when some agent uses more tokens than others? Also, how do you design the token-to-contribution denomination? Please help to clarify.\n2. Collaborative fairness is included as a major contribution of the proposed method. This notion is undefined in the paper itself. One possible place to find reference is [1], which is defined to be a high correlation between the performance of models received by the clients and the standalone performance of the client's local data. Can you produce a result that compares the collaborative fairness of various methods?\n3. The scenarios that the proposed method work seem to be too specialized and extreme. For example in Figure 4, PI-FL does not work so well in the 30:70 partition. It seems that a very extreme heterogeneity is needed to see the superiority of the proposed method, how to then tackle the case when some agents have very heterogeneous data while the rest of the agents have rather balanced datasets?\n4. The empirical results in Section 5.3 is not very convincing given the relatively easy dataset and limited baselines used. How about FedGroup and [29], are they being compared anywhere in terms of the clustering effectiveness?\n5. Does Table 1 and 2 show average accuracy? How many clients are there? Is the number of clients not so important in this experiment?\n6. Why is CIFAR-10 only tested on FedSoft and CIFAR-10? Can you provide a more complete comparison in the Appendix?\n\n**Minor**\n\n7. Some Line references for the Algorithms in Section 3 first paragraph seem to be incorrect.\n8. Should you add an indicator function in Equation (15)?\n\nLIMITATIONS:\n1. It might be necessary to discuss the incentive and fairness more clearly. For example, we can see from Figure 2 that clients have varying PMA, which indicates different levels of benefits were gained by different clients in the collaboration. How should this phenomenon be viewed under the lens of fairness? Also, the opt-outs in a very coarse binary indicator and might not be a good way to see whether clients are really incentivized.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

comments: SUMMARY:
In this paper, the authors propose a novel clustering-based pFL combined with a token-based incentive mechanism to address incentive provision in pFL. Specifically, the proposed method clusters clients based on their cluster preference (the distribution similarity), which maximizes their contribution to the clusters.

STRENGTHS:
1. The proposed approach of incentive-driven clustering based pFL is novel.

2. Theoretical analysis of the proposed method is provided.

3. Experimental results on multiple datasets demonstrate promising outcomes.

WEAKNESSES:
1. Some notations are not explained:


	What's $\zeta^{*}_k$ in Eqn.(4)? Only $\zeta_k$ is explained in line 177 of page 5

	What's $\theta_i$ in line 211 of page 6? Is it the same as the $\theta$ in Eqn.(6)?


2. Some parts of the method is not explained well


	What's the difference between reimbursement and payment? 

	What does the term of "token" actually means in pFL?

	What's the bidding strategy of clients?

3. Experiment:
	
	In pFL, the data are often heterogeneous in label shifting or feature shifting. However, the datasets used in the paper seems only related to dataset size imbalance, which are not the representative datasets used in pFL research. 

	Hope to see the experimental results on CIFAR10 data with Dirichlet splitting (referring the settings in [1]).

	[1] Marfoq, Othmane, et al. "Federated multi-task learning under a mixture of distributions." Advances in Neural Information Processing Systems 34 (2021): 15434-15447.

Minor:

	Typos: In figure 2, the x-axis is the number of clients which supposed to be a positive int type, but in the figure, it ranges from 0 to 1.

QUESTIONS:
1. Refer weakness 1 and weakness 2.

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents PI-FL, a new cluster-based personalized Federation Learning (pFL) approach. The new idea is the first to propose to consider motivation and personalization as interrelated challenges and address them through incentive mechanisms that promote personalized learning. PI-FL let clients provide incentive-driven preferences for joining clusters based on their own data distribution, and this client-centric clustering approach ensures accurate clustering and improved performance. This approach results in improved personalized model appeal (PMA) and reduced opt-outs, which in turn improves the accuracy of the clustering model. Theoretical analysis initially shows the effectiveness of the motivating algorithm. Experiments on several datasets show that the algorithm can obtain higher accuracy than the baseline.\n\nSTRENGTHS:\n1. This paper proposes to consider motivation and personalization as interrelated challenges and to address them through incentives that promote personalized learning. This is a good and reasonably innovative point that is informative for subsequent research on personalized federal learning.\n2. The ability of PI-FL, a customer-centric clustering approach, to access the original customer data ensures accurate clustering and improved performance even in the case of dynamic distribution shifts in the customer's local data or incorrect clustering decisions by the customer, which improves the robustness of federation learning.\n3. This paper is basically written very clearly. The three main modules of PI-FL: the profiler, the token manager, and the scheduler are very cleverly designed and their functions and working principles are well described.\n4. The theoretical analysis is carried out in the paper to prove the effectiveness of the incentive algorithm, and the theoretical part is relatively complete.\n5. The experimental comparison of test accuracy in multiple partitions shows that PI-FL can maintain good performance in all partitions. The comparison experiments of the two metrics, PMA and opt-outs, show that PI-FL can not only reduce opt-outs but also improve PMA in all data heterogeneity conditions.\n\nWEAKNESSES:\n1. The paper seems to lack the analysis about the convergence of PI-FL and the comparison with other pFL algorithms about the convergence speed.\n\n~~2. In order to show the effectiveness of PI-FL method, the comparison between PI-FL and other clustering-based pFL algorithms about computation and communication cost should be added, and there should be more comparisons between several clustering-based pFL algorithms instead of only comparing with FedSoft.~~\n\n3. All experiments are conducted in a simple CNN model, and the empirical study would be stronger if the authors could add some linguistic tasks, such as BERT pre-training/ fine-tuning experiments\n4. Figure 1, as the overall design of PI-FL, is not drawn in enough detail, e.g., the icons representing customers are not labeled as customers. the fold lines in Figure 4 are dense and affect the perception, e.g., it is difficult to see clearly the fold lines representing 10:90 (NI).\n5. The N_p parameter of the scheduler in PI-FL is the number of customers selected based on performance, and N_r is the number of customers selected randomly. N_p and N_r  parameters are very important for the training results of PI-FL, and further analysis should be done on how to select the appropriate N_p and N_r  parameters.\n\nQUESTIONS:\n1.The convergence analysis about PI-FL is missing.\n2. No runtime results are given in the experiments. Does PI-FL actually bring any speedup over other pFL methods? Are the actual computational and communication costs of PI-FL in an acceptable range?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}