SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 2 Payment-free Incentive Mechanism
Require:  $D^{p} = \{D_{i}^{p}|i\in [N]\}$ $\widetilde{S}_t = \{\Delta V_{i,t}|i\in [N]\}$    
1: Initialize participant set  $S_{t} = \widetilde{S}_{t}$    
2: while  $S_{t}\neq \emptyset$  do iteratively update  $S_{t}$  until it becomes stable   
3: StableFlag  $\equiv$  True   
4: for  $i:\Delta V_{i,t}\in S_t$  do   
5: if  $\mathcal{I}_{i,t} <   D_i^p$  then Eq. 4   
6: Update participant set  $S_{t} = S_{t}\setminus \{\Delta V_{i,t}\}$  remove client  $j$  from  $S_{t}$    
7: StableFlag  $\equiv$  False   
8: break   
9: if StableFlag  $\equiv$  True then   
10: break   
11: return  $S_{t}\subseteq \widetilde{S}_{t}$
4.2 Payment-free Incentive Mechanism
As mentioned in Section 1, in federated bandit learning, clients can reduce their regret by using models constructed via shared data. Denote  $\widetilde{V}_t$  as the covariance matrix constructed by all available data in the system at time step  $t$ . Based on Lemma 5 and 7, the instantaneous regret of client  $i_t$  is upper bounded by:
$$
r _ {t} \leq 2 \alpha_ {i _ {t}, t - 1} \sqrt {\mathbf {x} _ {t} ^ {\top} \widetilde {V} _ {t - 1} ^ {- 1} \mathbf {x} _ {t}} \cdot \sqrt {\frac {\operatorname* {d e t} (\widetilde {V} _ {t - 1})}{\operatorname* {d e t} (V _ {i _ {t} , t - 1})}} = O \left(\sqrt {d \log \frac {T}{\delta}}\right) \cdot \| \mathbf {x} _ {t} \| _ {\widetilde {V} _ {t - 1} ^ {- 1}} \cdot \sqrt {\frac {\operatorname* {d e t} (\widetilde {V} _ {t - 1})}{\operatorname* {d e t} (V _ {i _ {t} , t - 1})}} \tag {3}
$$
where the determinant ratio reflects the additional regret due to the delayed synchronization between client  $i_t$ 's local sufficient statistics and the global optimal oracle. Therefore, minimizing this ratio directly corresponds to reducing client  $i_t$ 's regret. For example, full communication keeps the ratio at 1, which recovers the regret of the centralized setting discussed in Section 3.1.
Therefore, given the client's desire for regret minimization, the data itself can be used as a form of incentive by the server. And the star-shaped communication network also gives the server an information advantage over any single client in the system: a client can only communicate with the server, while the server can communicate with every client. Therefore, the server should utilize this advantage to create incentives (i.e., the LHS of Eq. (1)), and a natural design to evaluate this data incentive is:
$$
\mathcal {I} _ {i, t} := \mathcal {I} _ {i, t} ^ {d} = \frac {\det \left(D _ {i , t} \left(S _ {t}\right) + V _ {i , t}\right)}{\det \left(V _ {i , t}\right)} - 1. \tag {4}
$$
where  $D_{i,t}(S_t) = \sum_{j:\{\Delta V_{j,t}\in S_t\} \wedge \{j\neq i\}}\Delta V_{j,t} + \Delta V_{-i,t}$  denotes the data that the server can offer to client  $i$  during the communication at time  $t$  (i.e., current local updates from other participants that have not been shared with the server) and  $\Delta V_{-i,t}$  is the historically aggregated updates stored in the server that has not been shared with client  $i$ . Eq. (4) suggests a substantial increase in the determinant of the client's local data is desired by the client, which ultimately results in regret reduction.
With the above data valuation in Eq. (4), we propose the payment-free incentive mechanism that motivates clients to share data by redistributing data collected from participating clients. We present this mechanism in Algorithm 2, and briefly sketch it below. First, we initiate the participant set  $S_{t} = \widetilde{S}_{t}$ , assuming all clients agree to participate. Then, we iteratively update  $S_{t}$  by checking the willingness of each client  $i$  in  $S_{t}$  according to Eq. (1). If  $S_{t}$  is empty or all clients in it are participating, then terminate; otherwise, remove client  $i$  from  $S_{t}$  and repeat the process.
While this payment-free incentive mechanism is neat and intuitive, it has no guarantee on the amount of data that can be collected. To see this, we provide a theoretical negative result with rigorous regret analysis in Theorem 3 (see proof in Appendix C).
Theorem 3 (Sub-optimal Regret) When there are at most  $\frac{c}{2C}\frac{N}{\log(T / N)}$  number of clients (for some constant  $C, c > 0$ ), whose cost value  $D_{i}^{p} \leq \min \{(1 + \frac{L^{2}}{\lambda})^{T}, (1 + \frac{T L^{2}}{\lambda d})^{d}\}$ , there exists a linear bandit instance with  $\sigma = L = S = 1$  such that for  $T \geq Nd$ , the expected regret for INC-FEDUCB algorithm with payment-free incentive mechanism is at least  $\Omega(d\sqrt{NT})$ .
Algorithm 3 Payment-efficient Incentive Mechanism
Require:  $\widetilde{S}_t = \{\Delta V_{i,t}|i\in [N]\}$ , data-incentivized participant set  $\widehat{S}_t\subseteq \widetilde{S}_t$ , threshold  $\beta$
1: for client  $i:\Delta V_{i,t}\in \tilde{S}_t\setminus \tilde{S}_t$  do:
2: Compute client's potential contribution to the server (i.e., marginal gain in determinant):
$$
c _ {i, t} (\widehat {S} _ {t}) = \det  \left(\Delta V _ {i, t} + V _ {g, t} (\widehat {S} _ {t})\right) / \det  \left(V _ {g, t} (\widehat {S} _ {t})\right), \quad V _ {g, t} (S _ {t}) = V _ {g, t - 1} + \Sigma (S _ {t}) \tag {6}
$$
3: Rank clients  $\{i_1,\dots ,i_m\}$  by their potential contribution, where  $m = |\widetilde{S}_t\setminus \widehat{S}_t|$
4: Segment the list by finding  $\alpha = \min \{j\mid \frac{\operatorname*{det}(V_{g,t}(\widehat{S}_t) + \Delta V_{i_j,t})}{\operatorname*{det}(V_{g,t}(\widehat{S}_t))}\geq \beta ,\forall j\in [m]\}$
5:  $k = \alpha -1,\mathcal{I}_{\mathrm{last}}^{m} = D_{i_{\alpha}}^{p} - \mathcal{I}_{i_{\alpha},t}^{d}$
6: return participant set  $S_{t} =$  Heuristic Search  $(k, \mathcal{I}_{\mathrm{last}}^{m})$
Algorithm 4
Recall the discussion in Section 3.1, when there is no communication  $R_{T}$  is upper bounded by  $O(d\sqrt{NT})$ . Hence, in the worst-case scenario, the payment-free incentive mechanism might not motivate any client to participate. It is thus not a sufficient mechanism.
4.3 Payment-efficient Incentive Mechanism
To address the insufficiency issue, we further devise a payment-efficient incentive mechanism that introduces additional monetary incentives to motivate clients' participation:
$$
\mathcal {I} _ {i, t} := \mathcal {I} _ {i, t} ^ {d} + \mathcal {I} _ {i, t} ^ {m} \tag {5}
$$
where  $\mathcal{I}_{i,t}^{d}$  is the data incentive defined in Eq. (4), and  $\mathcal{I}_{i,t}^{m}$  is the real-valued monetary incentive, i.e., the payment assigned to the client for its participation. Specifically, we are intrigued by the question: rather than trivially paying unlimited amounts to ensure everyone's participation, can we devise an incentive mechanism that guarantees a certain level of client participation such that the overall regret is still nearly optimal but under acceptable monetary incentive cost?
Inspired by the determinant ratio principle discussed in Eq. (3), we propose to control the overall regret by ensuring that every client closely approximates the oracle after each communication round, which can be formalized as  $\operatorname*{det}(V_{g,t}) / \operatorname*{det}(\tilde{V}_t) \geq \beta$ , where  $V_{g,t} = V_{g,t-1} + \Sigma(S_t)$  is to be shared with all clients and  $\Sigma(S_t) = \sum_{j: \{\Delta V_{j,t} \in S_t\}} \Delta V_{j,t}$ . The parameter  $\beta \in [0,1]$  characterizes the chosen gap between the practical and optimal regrets that the server commits to. Denote the set of clients motivated by  $\mathcal{I}_{i,t}^d$  at time  $t$  as  $S_t^d$  and those motivated by  $\mathcal{I}_{i,t}^m$  as  $S_t^m$ , and thus  $S_t = S_t^m \cup S_t^d$ . At each communication round, the server needs to find the minimum  $\mathcal{I}_{i,t}^m$  such that pooling local updates from  $S_t$  satisfies the required regret reduction for the entire system.
Note that Algorithm 2 maximizes  $\mathcal{I}_{i,t}^{d}$ , and thus the servers should compute  $\mathcal{I}_{i,t}^{m}$  on top of optimal  $\mathcal{I}_{i,t}^{d}$  and resulting  $S_{t}^{d}$ , which however is still combinatorially hard. First, a brute-force search can yield a time complexity up to  $O(2^{N})$ . Second, different from typical optimal subset selection problems [17], the dynamic interplay among clients in our specific context brings a unique challenge: once a client is incentivized to share data, the other uninvolved clients may change their willingness due to the increased data incentive, making the problem even more intricate.
To solve the above problem, we propose a heuristic ranking-based method, as outlined in Algorithm 3. The heuristic is to rank clients by the marginal gain they bring to the server's determinant, as formally defined in Eq. (6), which helps minimize the number of clients requiring monetary incentives, while empowering the participation of other clients motivated by the aggregated data. This forms an iterative search process: First, we rank all  $m$  non-participating clients (Line 2-3) by their potential contribution to the server (with participant set  $S_{t}$  committed); Then, we segment the list by  $\beta$ , anyone whose participation satisfies the overall  $\beta$  gap constraint is an immediately valid choice (Line 4). The first client  $i_{\alpha}$  in the valid list and its payment  $\mathcal{I}_{\mathrm{last}}^{m} (\infty$  if not available) will be our last resort (Line 5); Lastly, we check if there exist potentially more favorable solutions from the invalid list (Line 6). Specifically, we try to elicit up to  $k = \alpha - 1$  ( $k = m$  if  $i_{\alpha}$  is not available) clients from the invalid list in  $n \leq k$  rounds, where only one client will be chosen using the same heuristic in each round. If having  $n$  clients from the invalid list also satisfies the  $\beta$  constraint and results in a reduced monetary
incentive cost compared to  $\mathcal{T}_{\mathrm{last}}^m$ , then we opt for this alternative solution. Otherwise, we will adhere to the last resort.
This Heuristic Search is detailed in Appendix A, and it demonstrates a time complexity of only  $O(N)$  in the worst-case scenarios, i.e.,  $n = m = N$ . Theorem 4 guarantees the sufficiency of this mechanism w.r.t communication and payment bounds.
Theorem 4 Under threshold  $\beta$  and clients' committed data sharing cost  $D^{p} = \{D_{1}^{p},\dots ,D_{N}^{p}\}$ , with high probability the monetary incentive cost of INC-FEDUCB satisfies
$$
M _ {T} = O \left(\max  D ^ {p} \cdot P \cdot N - \sum_ {i = 1} ^ {N} P _ {i} \cdot \left(\frac {\det  (\lambda I)}{\det  (V _ {T})}\right) ^ {\frac {1}{P _ {i}}}\right).
$$
where  $P_{i}$  is the number of epochs client  $i$  gets paid throughout time horizon  $T$ ,  $P$  is the total number of epochs, which is bounded  $P = O(Nd\log T)$  by setting communication threshold  $D_{c} = \frac{T}{N^{2}d\log T} - \sqrt{\frac{T^{2}}{N^{2}dR\log T}}\log \beta$ , where  $R = \left\lceil d\log (1 + \frac{T}{\lambda d})\right\rceil$ .
Henceforth, the communication cost satisfies
$$
C _ {T} = O \left(N d ^ {2}\right) \cdot P = O \left(N ^ {2} d ^ {3} \log T\right)
$$
Furthermore, by setting  $\beta \geq e^{-\frac{1}{N}}$ , the cumulative regret is
$$
R _ {T} = O \left(d \sqrt {T} \log T\right)
$$
The proof of theorem 4 can be found in Appendix D.
We simulate the incentivized federated bandit problem under various environment settings. Specifically, we create an environment of  $N = 50$  clients with cost of data sharing  $D^{p} = \{D_{1}^{p},\dots ,D_{N}^{p}\}$ , total number of iterations  $T = 5,000$ , feature dimension  $d = 25$ , and time-varying arm pool size  $K = 25$ . By default, we set  $D_{i}^{p} = D_{\star}^{p}\in \mathbb{R},\forall i\in [N]$ . Due to the space limit, more detailed results and discussions on real-world dataset can be found in Appendix E.
(a)  $D_{\star}^{p} = 1$
(b)  $D_{\star}^{p} = 10$
Figure 1: Comparison between payment-free vs. payment-efficient incentive designs. The results are averaged over 10 runs with standard deviation as the error bars.
(c)  $D_{\star}^{p} = 100$
5.1 Payment-free vs. Payment-efficient
We first empirically compared the performance of the payment-free mechanism (named as INC-FEDUCB-PF) and the payment-efficient mechanism INC-FEDUCB in Figure 1. It is clear that the added monetary incentives lead to lower regret and communication costs, particularly with increased  $D_{\star}^{p}$ . Lower regret is expected as more data can be collected and shared; while the reduced communication cost is contributed by reduced communication frequency. When less clients can be motivated in one communication round, more communication rounds will be triggered as the clients tend to have outdated local statistics.
(a) Accumulative Regret
(b) Communication Cost
(c) Payment Cost
Figure 2: Ablation study on heuristic search (w.r.t  $D_{\star}^{p} \in [1,10,100]$ ). The results are averaged over 10 runs with standard deviation as the error bars.
5.2 Ablation Study on Heuristic Search
To investigate the impact of different components in our heuristic search, we compare the full-fledged model INC-FEDUCB with following variants on various environments: (1) INC-FEDUCB (w/o PF): without payment-free incentive mechanism, where the server only use money to incentivize clients; (2) INC-FEDUCB (w/o IS): without iterative search, where the server only rank the clients once. (3) INC-FEDUCB (w/o PF + IS): without both above strategies.
In Figure 2, we present the averaged learning trajectories of regret and communication cost, along with the final payment costs (normalized) under different  $D_{\star}^{p}$ . The results indicate that the full-fledged INC-FEDUCB consistently outperforms all other variants in various environments. Additionally, there is a substantial gap between the variants with and without the PF strategy, emphasizing the significance of leveraging the server's information advantage to motivate participation.
5.3 Environment & Hyper-Parameter Study
We further explored diverse  $\beta$  hyper-parameter settings for INC-FEDUCB in various environments with varying  $D_{\star}^{p}$ , along with the comparison with DisLinUCB [40] (only comparable when  $D_{\star}^{p} = 0$ ). Specifically, we explored different hyper-parameter settings for INC-FEDUCB with determinant ratio threshold  $\beta \in [0.3, 0.7, 1]$ , and various environmental configurations with data sharing cost  $D_{\ast}^{p} \in [1, 10, 100]$ .
d=25,K=25	DisLinUCB	INC-FEDUCB (β=1)	INC-FEDUCB (β=0.7)	INC-FEDUCB (β=0.3)
T=5,000,N=50,Dp=0	Regret (Acc.)	48.46	48.46	48.46 (Δ=0%)	48.46 (Δ=0%)
Commu. Cost	7,605,000	7,605,000	7,605,000 (Δ=0%)	7,605,000 (Δ=0%)
Pay. Cost	\	0	0 (Δ=0%)	0 (Δ=0%)
T=5,000,N=50,Dp=1	Regret (Acc.)	\	48.46	47.70 (Δ-1.6%)	48.38 (Δ-0.2%)
Commu. Cost	\	7,605,000	7,668,825 (Δ+0.8%)	7,733,575 (Δ+1.7%)
Pay. Cost	\	75.12	60.94 (Δ-18.9%)	22.34 (Δ-70.3%)
T=5,000,N=50,Dp=10	Regret (Acc.)	\	48.46	48.21 (Δ-0.5%)	47.55 (Δ-1.9%)
Commu. Cost	\	7,605,000	7,779,425 (Δ+2.3%)	8,599,950 (Δ+13%)
Pay. Cost	\	12,819.61	9,050.61 (Δ-29.4%)	4,859.17 (Δ-62.1%)
T=5,000,N=50,Dp=100	Regret (Acc.)	\	48.46	48.22 (Δ-0.5%)	48.44 (Δ-0.1%)
Commu. Cost	\	7,605,000	7,842,775 (Δ+3.1%)	8,718,425 (Δ+14.6%)
Pay. Cost	\	190,882.45	133,426.01 (Δ-30.1%)	88,893.78 (Δ-53.4%)
Table 1: Study on hyper-parameter of INC-FEDUCB and environment.
As shown in Table 1, when all clients are incentivized to share data, our INC-FEDUCB essentially recover the performance of DisLinUCB, while overcoming its limitation in incentivized settings when clients are not willing to share by default. Moreover, by reducing the threshold  $\beta$ , we can substantially save payment costs while still maintaining highly competitive regret, albeit at the expense of increased communication costs. And the reason for this increased communication cost has been explained before: more communication rounds will be triggered, as clients become more outdated.
In this work, we introduce a novel incentivized communication problem for federated bandits, where the server must incentivize clients for data sharing. We propose a general solution framework INC-FEDUCB, and initiate two specific implementations introducing data and monetary incentives, under the linear contextual bandit setting. We prove that INC-FEDUCB flexibly achieves customized levels of near-optimal regret with theoretical guarantees on communication and payment costs. Extensive empirical studies further confirmed our versatile designs in incentive search across diverse environments. Currently, we assume all clients truthfully reveal their costs of data sharing to the server. We are intrigued in extending our solution to settings where clients can exhibit strategic behaviors, such as misreporting their intrinsic costs of data sharing to increase their own utility. It is then necessary to study a truthful incentive mechanism design.
Acknowledgement. We thank the anonymous reviewers for their insightful and constructive comments. This project is partially supported by NSF Award IIS-2213700 and IIS-2128019. Haifeng Xu is supported by an NSF Award CCF-2303372, an Army Research Office Award W911NF-23-1-0030, and an Office of Naval Research Award N00014-23-1-2802.
[1] Yasin Abbasi-Yadkori, David Pal, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In NIPS, volume 11, pages 2312-2320, 2011.
[2] Maurice Allais. Le comportement de l'homme rationnel devant le risque: critique des postulats et axiomes de l'école française. *Econometrica: Journal of the econometric society*, pages 503-546, 1953.
[3] Nicolo Cesa-Bianchi, Claudio Gentile, and Giovanni Zappella. A gang of bandits. In Advances in Neural Information Processing Systems, pages 737-745, 2013.
[4] Mithun Chakraborty, Kai Yee Phoebe Chua, Sanmay Das, and Brendan Juba. Coordinated versus decentralized exploration in multi-agent multi-armed bandits. In *IJCAI*, pages 164–170, 2017.
[5] Yae Jee Cho, Divyansh J Hunjhunwala, Tian Li, Virginia Smith, and Gauri Joshi. To federate or not to federate: Incentivizing client participation in federated learning. In Workshop on Federated Learning: Recent Advances and New Challenges (in Conjunction with NeurIPS 2022), 2022.
[6] Jiu Ding and Aihui Zhou. Eigenvalues of rank-one updated matrices with some applications. Applied Mathematics Letters, 20(12):1223-1226, 2007.
[7] Kate Donahue and Jon Kleinberg. Model-sharing games: Analyzing federated learning under voluntary participation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5303-5311, 2021.
[8] Kate Donahue and Jon Kleinberg. Fairness in model-sharing games. In Proceedings of the ACM Web Conference 2023, pages 3775-3783, 2023.
[9] Yihan Du, Wei Chen, Yuko Kuroki, and Longbo Huang. Collaborative pure exploration in kernel bandit. In The Eleventh International Conference on Learning Representations, 2023.
[10] Abhimanyu Dubey and AlexSandy' Pentland. Differentially-private federated linear bandits. Advances in Neural Information Processing Systems, 33:6003-6014, 2020.
[11] Sarah Filippi, Olivier Cappe, Aurélien Garivier, and Csaba Szepesvári. Parametric bandits: The generalized linear case. Advances in Neural Information Processing Systems, 23, 2010.
[12] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1-19, 2015.
[30] Jian Pei. A survey on data pricing: from economics to data science. IEEE Transactions on knowledge and Data Engineering, 34(10):4586-4608, 2020.
[31] Malcolm Pemberton and Nicholas Rau. Mathematics for economists: an introductory textbook. Manchester University Press, 2007.
[32] Abishek Sankararaman, Ayalvadi Ganesh, and Sanjay Shakkottai. Social learning in multi agent multi armed bandits. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 3(3):1-35, 2019.
[33] Chengshuai Shi, Cong Shen, and Jing Yang. Federated multi-armed bandits with personalization. In International Conference on Artificial Intelligence and Statistics, pages 2917–2925. PMLR, 2021.
[34] Chengshuai Shi, Wei Xiong, Cong Shen, and Jing Yang. Decentralized multi-player multi-armed bandits with no collision information. In International Conference on Artificial Intelligence and Statistics, pages 1519–1528. PMLR, 2020.
[35] Rachael Hwee Ling Sim, Yehong Zhang, Mun Choon Chan, and Bryan Kian Hsiang Low. Collaborative machine learning with incentive-aware model rewards. In International conference on machine learning, pages 8927-8936. PMLR, 2020.
[36] Balazs Szorenyi, Róbert Busa-Fekete, István Hegedus, Róbert Ormandi, Márk Jelasy, and Balázs Kégl. Gossip-based distributed stochastic bandit algorithms. In International conference on machine learning, pages 19-27. PMLR, 2013.
[37] Chao Tao, Qin Zhang, and Yuan Zhou. Collaborative learning with limited interaction: Tight bounds for distributed exploration in multi-armed bandits. In 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS), pages 126–146. IEEE, 2019.
[38] Xuezhen Tu, Kun Zhu, Nguyen Cong Luong, Dusit Niyato, Yang Zhang, and Juan Li. Incentive mechanisms for federated learning: From economic and game theoretic perspective. IEEE Transactions on Cognitive Communications and Networking, 2022.
[39] Po-An Wang, Alexandre Proutiere, Kaito Ariu, Yassir Jedra, and Alessio Russo. Optimal algorithms for multiplayer multi-armed bandits. In International Conference on Artificial Intelligence and Statistics, pages 4120–4129. PMLR, 2020.
[40] Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, and Liwei Wang. Distributed bandit learning: Near-optimal regret with efficient communication. In International Conference on Learning Representations, 2020.
[41] Xinyi Xu, Lingjuan Lyu, Xingjun Ma, Chenglin Miao, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Gradient driven rewards to guarantee fairness in collaborative machine learning. Advances in Neural Information Processing Systems, 34:16104-16117, 2021.
[42] Yufeng Zhan, Jie Zhang, Zicong Hong, Leijie Wu, Peng Li, and Song Guo. A survey of incentive mechanism design for federated learning. IEEE Transactions on Emerging Topics in Computing, 10(2):1035-1044, 2021.
[43] Zhaowei Zhu, Jingxuan Zhu, Ji Liu, and Yang Liu. Federated bandit: A gossiping approach. In Abstract Proceedings of the 2021 ACM SIGMETRICS/International Conference on Measurement and Modeling of Computer Systems, pages 3-4, 2021.
A Heuristic Search Algorithm
Algorithm 4 Heuristic Search
Require: invalid client list  $\{i_1,i_2,\dots ,i_k\}$  data-incentivized participant set  $\widehat{S}_t$  and the last resort cost  $\mathcal{I}_{\mathrm{last}}^m$    
1: Initialization:  $S_{t} = \widehat{S}_{t}$    
2: for  $n\in [k]$  do   
3: Rank clients  $\{i_1,\ldots ,i_{k - n + 1}\}$  (in new order) by Eq (6)   
4:  $S_{t} = S_{t}\cup \{i_{k - n + 1}\}$  add the client with the largest contribution   
5: for client  $j\in \{i_1,\dots ,i_{k - n}\}$  do find extra data-incentivized participants   
6: Compute data incentive  $\mathcal{I}_{j,t}^{d}$  for client  $j$  by Eq (4)   
7: if  $\mathcal{I}_{j,t}^{d} > D_{j}^{p}$  then   
8:  $S_{t} = S_{t}\cup \{\Delta V_{j,t}\}$    
9: Compute total payment  $\mathcal{I}_{n,t}^{m} = \sum_{i\in \widetilde{S}_{t}\backslash S_{t}}\mathcal{I}_{i,t}^{m}$  by Eq (5)   
10: if  $\mathcal{I}_{n,t}^{m}\leq \mathcal{I}_{\mathrm{last}}^{m}$  then   
11: return  $S_{t} = \widehat{S}_{t}\cup \{\Delta V_{i_{\alpha},t}\}$  return last resort   
12: else   
13: if det(  $(\Sigma (S_{t}) + V_{g,t - 1}) / \det (\Sigma (\widetilde{S}_{t}) + V_{g,t - 1}) > \beta$  then   
14: return  $S_{t}$  return search result