# Federated Learning Incentive Mechanism Design via Enhanced Shapley Value Method

Authors: Xun Yang; Weijie Tan; Changgen Peng; Shuwen Xiang; Kun Niu
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

Federated learning (FL) is an emerging collaborative machine learning method. In FL processing, the data quality shared by users directly affects the accuracy of the federated learning model, and how to encourage more data owners to share data is crucial. In other words, how to design a good incentive mechanism is the key problem in FL. In this paper, we propose an incentive mechanism based on the enhanced Shapley value method for FL. In the proposed mechanism, the enhanced Shapley value method is proposed to measure income distribution, which takes multiple influence factors as weights. The analytic hierarchy process (AHP) is used to find the corresponding weight value of the influence factors. Finally, the numerical experiments are carried to verify the performance of the proposed incentive mechanism. The results show that compared with the Shapley value method considering the single factor, the income distribution of all participants can better reflect multiple factor contribution when using the enhanced Shapley value method.


---

## Body

## 1. Introduction

With the rapid development of the artificial intelligence technology, it is quite common for cross-organizational and cross-institutional data use. How to ensure data privacy and security while realizing data sharing [1, 2] have attracted extensive attention, which is a fundamental challenge in the era of artificial intelligence [3–6]. Federated learning (FL) is a new distributed machine learning method to address the data island problem, and it was first proposed by Google in 2016 [7–9], which can be applied to the Android phone user’s local model updating and training between multiple participants or multiple computing nodes. Nowadays, it has been used in all walks of life widely.In the process of FL, the incentive mechanism is an important research topic [10–12]. To make all data owners contribute their data for model training actively, it is necessary to establish a reasonable incentive mechanism to encourage data owners to share valuable data. A valuable incentive mechanism can enable all participants to continuously and carry out efficient model training and improve the final trained federated model more accuracy. When there is no a meaningful incentive mechanism, all participants do not update data in real time in the process of data federation and do not contribute to FL actively, which will not obtain the accurate training model, and all participants dare not use the trained model. It will also cause a waste of time and economic investment of all participants.Recently, there are a series of research results about the incentive mechanism of FL. In [11], the authors proposed an incentive scheme for a local, client machine learning model for FL, which solves the problem that participants may opt out of the federated system and provides useless information to the federated system without satisfactory incentives. To facilitate collaborative machine learning among multiple model owners, the work in [12] proposed a hierarchical incentive mechanism framework, which solves the problem of hierarchical structure within federated systems. Zhan et al. [13] designed a deep reinforcement learning-based incentive mechanism, which solves unique the nonsharing of data and difficult contribution evaluation. Yu et al. [14] proposed a divide budget incentive method, which solves the restriction of temporary mismatch between contribution and reward in FL. From these researches, we note that participant’s submodel influences the quality of trained model generated from FL. Generally, the participants have an enthusiasm to cooperate with each other for improving the quality of model. In [15, 16], all participants in FL are considered as having equivalent contributions. However, the contributions of participants have distinct in some cases. For example, the FL model is applied in WeBank, which has cooperated with such financial institutions, insurance companies, and enterprises.In these existing works, they do not take multiple distribution factors such as cost input, risk factor, and data quality that affect the distribution of benefits. To obtain income distribution fair and just, we proposed an incentive mechanism based on the enhanced Shapley value method for FL. In the proposed method, all federated participants get different federated incomes according the contribution in model training, and there is no free-riding among all participants.The major contributions of this paper are summarized as the threefold:(1)
We construct the federated incentive model for multiple participants, introduce the cooperative game to the incentive model in FL, and consider multiple factors that affect the income distribution of participants in the FL system(2)
To allocate the income for the federated participants more fair, we propose the analytic hierarchy process (AHP) to construct the expert judgment matrix according the multiple contribution factors, which achieves the corresponding weights of all the influencing factors(3)
The effectiveness of this incentive mechanism is verified through numerical experiments. Experimental results show that the income distribution of all participants can better reflect multiple factor contributionThe rest of this paper is organized as follows: The related work and our main contribution of this paper are introduced in Section2. The definition of FL, cooperative games, Shapley value, and the AHP are introduced in Section 3. The fair federated model and incentive enhanced Shapley value method are established in Section 4. The rationality of the method is verified by numerical experiments and numerical comparison in Section 5. The conclusion and future work are drawn in Section 6.


## 2. Related Work

The model training accuracy of FL is affected by incentive mechanism, which has been widely concerned at present [15–17]. In [18, 19], the authors proposed an incentive mechanism which combined the contract and reputation can improve the efficiency of federated model training. In [20], an optimization method in the FL wireless networks was formulated, which captures two trade-offs and obtains the global optimal solution by characterizing the closed-form solutions to all subproblems. In [21, 22], within the Stackelberg framework, a motivation-based interaction model was established between global servers and participating devices to encourage participants taking part in cooperation. Based on fair allocation for wireless network resources, Li et al. [23] proposed a new objective of resource optimization, which promote a fair society from the perspective of fairness, flexibility, and efficiency. For the linear regression model, the work [24] designed an incentive payment structure to encourage agents provide high-quality data, which can describe the impact of data points on the loss function of the model. The game framework was proposed in [25, 26] to solve the efficiency problem of multiparty model.In the incentive mechanism, the return fairness is a worthy studying problem. The work of [27] studied the dynamics of coalition formation under the bounded rationality condition, which considers the situation that the profit of each team is given by the submodular function, and proposed a profit distribution scheme based on the concept of marginal utility. Xiong et al. [28] proposed an incentive mechanism for the multiattribute user selection, which effectively improves sensing user quality for mobile crowdsensing. The Shapley value method is used to distribute fair the benefits generated in the process of the coalition [29], which is the most widely used method to evaluate fairness [30, 31]. In [32], the authors combined the quality estimation with a monetary incentive and used the Shapley value method to determine the cost of each household, which was based on the effect of goodness in saving and remaining points. In [33], a repertoire of efficient algorithms was proposed to meet the time-consuming challenge in the Shapley value requiring process. Liu et al. [34] proposed a blockchain-based FedCoin peer-to-peer payment incentive mechanism for FL and used the Shapley value method to distribute federated revenue.In the existing works, they do not take multiple related factors such as cost input, risk factor, and data quality that affect the distribution of benefits. In order to fully consider the impact of multiple factors on the fairness of distribution, an incentive mechanism for the FL system is proposed to achieve fairness and justice, which uses the enhanced Shapley value method. The proposed incentive mechanism introduces the cooperative game to the incentive model and considers multiple factors that affect the income distribution of participants. The analytic hierarchy process (AHP) is used to construct the expert judgment matrix, which achieves the corresponding weights of all the influencing factors.


## 3. Preliminaries

### 3.1. Definition of Federated Learning

Let1,2,⋯,N be defined as N data owners, and they all hope to merge their respective data D1,D2,⋯,DN to train a machine learning model. A model MSUM is trained by a traditional method, which puts all data D=D1∪D2∪⋯∪DN together. A model MFED can be cooperatively trained by the data owners of claimed FL system in learning process, which the data Di never leaked to other participant via any data owner i in this process. Moreover, the precision of MFED, defined as VFED, should be pretty close to the property of MSUM and VSUM. Generally, if there is a δ≥0 satisfies the condition
(1)VFED−VSUM<δ,we say that the FL algorithm hasδ-accuracy loss [4].


### 3.2. Cooperative Games

TheGN,v is defined cooperative game and satisfies two conditions as follows [29, 35, 36]:
(2)vS1+vS2≤vS1∪S2,(3)S1∩S2=∅,v∅=0,where N is a finite set of participants, S1,S2∈2N, v:2N⟶R is characteristic function, 2N is the set of all the subsets of N. Let vS is participants’ income function, vN indicates the coalition income and φiv (defined in formula (6)) is the income of participant i in vN, which satisfy two constraints:
(4)vN=∑i=1nφiv,(5)vS≤∑i∈Sφiv,Formula (4) is named as collective rationality, which indicates that the sum of all participants’ incomes from the maximum income in GN,v should be equivalent to vN, and formula (5) is claimed coalition rationality, which shows that the sum of profit allocated to all probable coalitions must be greater than or equivalent to vS. When S=1, the formula (5) is individual rationality.


### 3.3. Shapley Value

Shapley value was presented by Shapley [29, 37] in the cooperative game theory, which can effectively solve the problem of cooperative income distribution and define as the following equation:
(6)φiv=∑i∈S,S⊂NwSvS−vS\i,(7)wS=n−S!S−1!n!,where S⊂N, i=1,2,⋯,n, S is the quantity of participants in subset S, wS is weight coefficient, vS is the income of subset S and satisfies Equations (2) and (3), the term vS−vS\i evaluates the marginal contribution of i to the coalition S, and vS\i shows the benefit of other players in subset S other than i.


## 3.1. Definition of Federated Learning

Let1,2,⋯,N be defined as N data owners, and they all hope to merge their respective data D1,D2,⋯,DN to train a machine learning model. A model MSUM is trained by a traditional method, which puts all data D=D1∪D2∪⋯∪DN together. A model MFED can be cooperatively trained by the data owners of claimed FL system in learning process, which the data Di never leaked to other participant via any data owner i in this process. Moreover, the precision of MFED, defined as VFED, should be pretty close to the property of MSUM and VSUM. Generally, if there is a δ≥0 satisfies the condition
(1)VFED−VSUM<δ,we say that the FL algorithm hasδ-accuracy loss [4].


## 3.2. Cooperative Games

TheGN,v is defined cooperative game and satisfies two conditions as follows [29, 35, 36]:
(2)vS1+vS2≤vS1∪S2,(3)S1∩S2=∅,v∅=0,where N is a finite set of participants, S1,S2∈2N, v:2N⟶R is characteristic function, 2N is the set of all the subsets of N. Let vS is participants’ income function, vN indicates the coalition income and φiv (defined in formula (6)) is the income of participant i in vN, which satisfy two constraints:
(4)vN=∑i=1nφiv,(5)vS≤∑i∈Sφiv,Formula (4) is named as collective rationality, which indicates that the sum of all participants’ incomes from the maximum income in GN,v should be equivalent to vN, and formula (5) is claimed coalition rationality, which shows that the sum of profit allocated to all probable coalitions must be greater than or equivalent to vS. When S=1, the formula (5) is individual rationality.


## 3.3. Shapley Value

Shapley value was presented by Shapley [29, 37] in the cooperative game theory, which can effectively solve the problem of cooperative income distribution and define as the following equation:
(6)φiv=∑i∈S,S⊂NwSvS−vS\i,(7)wS=n−S!S−1!n!,where S⊂N, i=1,2,⋯,n, S is the quantity of participants in subset S, wS is weight coefficient, vS is the income of subset S and satisfies Equations (2) and (3), the term vS−vS\i evaluates the marginal contribution of i to the coalition S, and vS\i shows the benefit of other players in subset S other than i.


## 4. Federated Learning Incentive Mechanism

Before establishing the FL incentive model, we make the following assumptions:(1)
All participants have the ability to pay for FL, and in the process of income distribution, they adopt the best income distribution scheme.(2)
All participants are willing to participate in the coalition and do not withdraw from each sub-coalition, and all parties are very satisfied with the final income distribution scheme.(3)
All participants are completely trustworthy, and there is no cheating.(4)
FL should adopt multiparty agreement to recognize the income distribution scheme to ensure the smooth implementation of the strategy.


### 4.1. FL Incentive Model

The proposed FL incentive model is as shown in Figure1, and in this model, we assume that there are n factors and n participants and establish their federated models. We use different colors to represent the process of different factors for different participants, the direction of model updating, and income distribution in the whole FL process.Step 1.
Under the influence of factor 1, factor 2,…, and factorn on the federated income, we distribute model 1, model 2,…, and model n to participant 1, participant 2,…, and participant n and then train the n submodels respective of data participants. The data of the n participants do not leave the local, to protect the privacy of the data in their respective databases.Step 2.
Every time the model runs, the parameters of then submodels are updated, until the end of the model run, and the models are aggregating to get the federated aggregation model we need.Step 3.
Every time the model runs, using formula (19), each participant i will receive the income φ~iv,t from the federated expected to cost bt in round t.Step 4.
When the model is running finished, the parameters of all submodels are aggregating to get the final aggregated federated model. Finally, the federated income is allocated according to the enhanced Shapley value method, and each participanti will achieve income φi′v,t from formula (20).Figure 1
Federated learning incentive model.The income distribution of each participanti is affected by n factors, which are allocated to rely on the contribution of each participant i to the whole federation. This design makes participants get the distributed federated benefits more fairly and get an accurate federated model. Because the influencing factors impact income of participants, the enthusiasm of participants will be affected and, finally, impact the accuracy of the federated model. If each participant i actively cooperates in the federated process and tries to take part in the FL model to update data in the local database in real time, the parameters of the model training will also be updated, and the final federated aggregation model will also be updated. This cycle will continue until the end of the whole model training. Finally, the total federated income generated is distributed according to the enhanced Shapley value method.


### 4.2. Contribution Assessment

To encourage the participants to actively participate in the FL, we set each participant to be treated fairly based on their contribution and meet three fair conditions at the same time:(1)
Contribution Fairness. Each participant’s income is related to its contribution to the federation.(2)
Expectation Fairness of Risk-Taking. The expected risk and time risk between each participant are minimized.(3)
Expectation Fairness. The expected and time risks taken from each participant vary as little as possible.We denotecit the cost of participant i and qit the contribution, and qit≥0 includes risk-taking, data quality, effort degree, and cost input. And we shall use the procurement auction method [41] to estimate cit. Further, we define the expected risk fit and time risk git by the following dynamic system:
(8)fit+1=maxfit+cit−φi′v,t,0,git+1=maxgit+μit−φi′v,t,0,where μit represents an indicator function is
(9)μit=c~it,fit>0,0,fit≤0.Here, whenfit>0, the time series git will increase, and c~it shows that participant i is the average cost of contribution for the federation. So we have a federally maximized H.
(10)H=1T∑t=1T∑i=1Nqitφi′v,t,satisfying fair condition (1), where T is the total quantity of runs of the model and qit represents the contribution made by participant i to the federation. We should treat all participants fairly to minimize their expected loss and distribution. Therefore, we introduce the Lyapunov function formulated as follows [42]:
(11)Lt=12∑i=1Nfi2t+gi2t.For the convenience of calculation, the root operator is omitted calculation, which will not change the good properties of formula (11). With the change of time, the expected risk deviation of the participants is as follows:
(12)ΔLt=1T∑t=1TLt+1−Lt=1T∑t=1T∑i=1N12fi2t+1−12fi2t+12gi2t+1−12gi2t≤1T∑t=1T∑i=1Nfitcit−fitφi′v,t+12ci2t−citφi′v,t+12φi′2v,t+gitμit−gitφi′v,t+12μi2t−μitφi′v,t+12φi′2v,t.Sinceφi′v,t is a control variable of the income function of the participant i, we extract the formula containing it from formula (12) and get the following result:
(13)ΔLt≤1T∑t=1T∑i=1Nφi′2v,t−φi′v,tfit+cit+git+μit.The expected lossΔLt is the distribution function of fit and git. If ΔLt is minimized and fair conditions (2) and (3) are satisfied, the expected loss will gradually decrease with time. Let
(14)Ft=εH−ΔLt,is the target number, which indicates the inequality of expected loss and waiting time between participants, and it should be minimized, where ε is a federated regularization term to constraint the equilibrium between the two targets. Therefore, the federation is aimed at the following function:
(15)Ft=1T∑t=1T∑i=1Nφi′v,tεqit+fit+cit+git+μit−φi′2v,t,and the constraint condition is
(16)∑i=1Nφ~iv,t≤bt,φ~iv,t≥0,for∀i,t,where bt means the cost of participants in the current payment budget, which is determined by their expected loss and the time they spend waiting for the reward to be fully paid, and the φ~it represents the income that participant i received from the federation in round t. Let us take the derivative of formula (15) with respect to φi′v,t, and set the first derivative equal to 0 to obtain φi′v,t:
(17)φi′v,t=12εqit+fit+cit+git+μit.Moreover, by using the second derivatives of (15) with respect to φi′t, we obtain the following result:
(18)d2φi′2v,tFt=−1<0.When the federated model runs in the roundt, the contribution of the participants to the federation is qit, and the total compensation of the participants i is φi′v,t=1/2εqit+fit+cit+git+μit. If budget bt is not enough to fully pay all participants’ compensation in round i, the federated organization will pay them in installments within a certain period. The federated organization will pay φ~iv,t in installments in round i according to the following equation:
(19)φ~iv,t=φi′v,t∑i=1Nφi′v,tbt.For the income distribution scheme in Figure1, when fit and git are equal to 0, this is a very important condition, and participant i does not invest extra cost; then, φi′v,t=εqit. After that, participant i will use the enhanced Shapley value method to evaluate its contribution based on the federation and allocate the future income. This income allocation method will first consider the participants whose expected risk is not equal to 0 and also consider the contributions of other participants to the federation. According to Figure 1 and the above theoretical analysis, we present the pseudo-code of FL incentive in Algorithm 1.Algorithm 1: FL incentive mechanism via enhanced Shapley value method.
Input:ε and bt are given by the federated system administrator. fit and git) are the expected risk of all participants and the income of all participants respectively in round t.Output:φ~1v,t,φ~2v,t,⋯,φ~Nv,t1: Initializationht⟵0; To stores the sum of all φi′v,t values.2:fori=1,2,⋯,Ndo3:ifdit>0then; dit represents the contribution scale value of participant i to the federated model.4:  Calculationcit;5:  Calculationqit;6:else7:cit=0;8:φi′v,t⟵12εqit+fit+cit+git+μit;9:ht⟵ht+φi′v,t;10:ifi=1,2,⋯,Nthen11:φ~iv,t⟵φi′v,thtbt12:fit+1=maxfit+cit−φ~iv,t,0;13:git+1=maxgit+μit−φ~iv,t,0;14:returnφ~1v,t,φ~2v,t,⋯,φ~Nv,t


### 4.3. The Enhanced Shapley Value Method

To ensure that all participants are satisfied with the federated income distribution scheme and make the distribution mechanism play an incentive role, all participants can actively contribute to the federated model. The Shapley value method considers that the contribution degree of each participant as1/n ignores the influence of other factors. Therefore, it is necessary to consider the effects of multiple factors on federal income distribution, by taking multiple influence factors as weights, and the enhanced Shapley value method is proposed to measure income distribution [28, 40].
(20)φi′v,t=φiv,t+vN×ΔPi,ΔPi=Pi−1n,i=1,2,⋯,n,(21)Pi=αi1,αi2,⋯,αinw1,w2,⋯,wnT,i=1,2,⋯,n,where φi′v,t shows the anticipated income for federated model i after improvement and under a federated game condition in the t round, vN is the revenue of the grand coalition, φiv,t is the income distribution of participant i using the Shapley value method in the t round of FL. In particular, when t=1, φi′v,t is calculated using formula 6, and ΔPi indicates the difference between the comprehensive evaluation factor Pi and the average factor 1/n introduced by participant i, and it refers to the comprehensive factor as an incentive factor in the federated distribution of incomes.
(22)ΔPi=Participantishouldbeencouraged,Pi−1n>0,Participantishouldbepunished,Pi−1n<0.Pi is the sum of the product of the measured values αi1, αi2,…, αin of the n factors and the corresponding factor weight w1, w2,…,wn, and ∑i=1nPi=1. Thus, we have
(23)∑i=1nΔPi=0,∑i∈SΔPi≥vS−∑i∈Sφiv,tvS.We solve these weights ofn influence factors αi1, αi2,…, αin with the AHP. It is easy to verify that formula (20) satisfies formulas (4) and (5).Proof.
According to formulas (4) and (23), formula (20) can be proofed as follows:
(24)∑i=1nφi′v,t=∑i=1nφiv,t+vN×ΔPi=∑i=1nφiv,t+vN∑i=1nΔPi=vN.Hence, formula (20) satisfies formula (4). In addition, according to formulas (5) and (23), formula (20) is proofed as the following:
(25)∑i∈Sφi′v,t=∑i∈Sφiv,t+vS×ΔPi=∑i∈Sφiv,t+vS∑i∈SΔPi≥vS.Hence, formula (20) satisfies formula (5).To solve the Shapley value correction factor of the federated income distribution, the AHP is used to assign the importance degree according to the scale of 1-9. The explanation of scale 1-9 is presented in Table1 [39].Table 1
The explanation of index in expert judgment matrix.
IndexExplanation1The factorsi and j are equally essential2The factori is more essential than j slightly3The factori is more essential than j obviously4The factori is more essential than j strongly5The factori is more essential than j extremely2, 4, 6, 8The index between adjacent indexes aboveReciprocalThe valueaij is obtained by factor i and factor j, and the value aji=1/aij is obtained by j comparing factor with factor iIn Table1, index indicates the intensity of importance on an absolute scale. The judgment matrix in this paper is constructed by the joint score of the expert group, and they score according to the rules in Table 1. These experts have a strong background in economics, computer, and mathematics. Hence, according to the joint score of the expert group, the judgment matrix Aaijn×n is obtained.
(26)A=a11a12⋯a1na21a22⋯a2n⋮⋮⋮⋮an1an2⋯ann,and satisfiedaij>0, aji=1/aij, and aii=1. The matrix Bbijn×n is given by normalizing, and according to each column of the matrix Aaijn×n, bij=aij/∑i,j=1naij, αi1, αi2,⋯, αin, comes from each i-th row element of matrix Bbijn×n and represents the measurement value of n factors affecting federated revenue; αij represents the j-th factor affecting the individual’s federated income, and ∑i,j=1nαij=1.
(27)B=b11b12⋯b1nb21b22⋯b2n⋮⋮⋮⋮bn1bn2⋯bnn.Sum matrixBbijn×n is according to row vector, ci=∑i,j=1nbij, and achieves the vector C.
(28)C=c1,c2,⋯,cn,and then, normalizing the vector C, wi=ci/∑i=1nci. Consequently, the weight value w of judgment matrix Aaijn×n is obtained.
(29)w=w1,w2,⋯,wnT,∑i=1nwi=1.Since the judgment matrixAaijn×n is established based on the experience and estimation of experts, some differences will appear between the judgment matrix A and the real situation, and it needs to test the consistency of the matrix Aaijn×n. If the matrix Aaijn×n is consistent, its eigenvector w=w1,w2,⋯,wnT can truly reflect the reasons that affect federated income. In this case, we can use the random consistency index (RI) [38, 39] in Table 2.
(30)λmax=∑i=1nAwinwi,CI=λmax−nn−1,CR=CIRI,where λmax is the largest eigenvalue of the matrix Aaijn×n, the CI represents public consistency index, RI is the average random consistency index, and CR is the consistency ratio. As we refer to Table 2, matrix Aaijn×n needs to be satisfied with the consistency.
(31)CR=ThematrixAaijn×nisjustwhatweneed,CIRI<0.1,ThematrixAaijn×nneedstobeadjusted,CIRI>0.1.Table 2
Random consistency index.
Order123456789RI0.000.000.520.891.111.251.351.41.45


## 4.1. FL Incentive Model

The proposed FL incentive model is as shown in Figure1, and in this model, we assume that there are n factors and n participants and establish their federated models. We use different colors to represent the process of different factors for different participants, the direction of model updating, and income distribution in the whole FL process.Step 1.
Under the influence of factor 1, factor 2,…, and factorn on the federated income, we distribute model 1, model 2,…, and model n to participant 1, participant 2,…, and participant n and then train the n submodels respective of data participants. The data of the n participants do not leave the local, to protect the privacy of the data in their respective databases.Step 2.
Every time the model runs, the parameters of then submodels are updated, until the end of the model run, and the models are aggregating to get the federated aggregation model we need.Step 3.
Every time the model runs, using formula (19), each participant i will receive the income φ~iv,t from the federated expected to cost bt in round t.Step 4.
When the model is running finished, the parameters of all submodels are aggregating to get the final aggregated federated model. Finally, the federated income is allocated according to the enhanced Shapley value method, and each participanti will achieve income φi′v,t from formula (20).Figure 1
Federated learning incentive model.The income distribution of each participanti is affected by n factors, which are allocated to rely on the contribution of each participant i to the whole federation. This design makes participants get the distributed federated benefits more fairly and get an accurate federated model. Because the influencing factors impact income of participants, the enthusiasm of participants will be affected and, finally, impact the accuracy of the federated model. If each participant i actively cooperates in the federated process and tries to take part in the FL model to update data in the local database in real time, the parameters of the model training will also be updated, and the final federated aggregation model will also be updated. This cycle will continue until the end of the whole model training. Finally, the total federated income generated is distributed according to the enhanced Shapley value method.


## 4.2. Contribution Assessment

To encourage the participants to actively participate in the FL, we set each participant to be treated fairly based on their contribution and meet three fair conditions at the same time:(1)
Contribution Fairness. Each participant’s income is related to its contribution to the federation.(2)
Expectation Fairness of Risk-Taking. The expected risk and time risk between each participant are minimized.(3)
Expectation Fairness. The expected and time risks taken from each participant vary as little as possible.We denotecit the cost of participant i and qit the contribution, and qit≥0 includes risk-taking, data quality, effort degree, and cost input. And we shall use the procurement auction method [41] to estimate cit. Further, we define the expected risk fit and time risk git by the following dynamic system:
(8)fit+1=maxfit+cit−φi′v,t,0,git+1=maxgit+μit−φi′v,t,0,where μit represents an indicator function is
(9)μit=c~it,fit>0,0,fit≤0.Here, whenfit>0, the time series git will increase, and c~it shows that participant i is the average cost of contribution for the federation. So we have a federally maximized H.
(10)H=1T∑t=1T∑i=1Nqitφi′v,t,satisfying fair condition (1), where T is the total quantity of runs of the model and qit represents the contribution made by participant i to the federation. We should treat all participants fairly to minimize their expected loss and distribution. Therefore, we introduce the Lyapunov function formulated as follows [42]:
(11)Lt=12∑i=1Nfi2t+gi2t.For the convenience of calculation, the root operator is omitted calculation, which will not change the good properties of formula (11). With the change of time, the expected risk deviation of the participants is as follows:
(12)ΔLt=1T∑t=1TLt+1−Lt=1T∑t=1T∑i=1N12fi2t+1−12fi2t+12gi2t+1−12gi2t≤1T∑t=1T∑i=1Nfitcit−fitφi′v,t+12ci2t−citφi′v,t+12φi′2v,t+gitμit−gitφi′v,t+12μi2t−μitφi′v,t+12φi′2v,t.Sinceφi′v,t is a control variable of the income function of the participant i, we extract the formula containing it from formula (12) and get the following result:
(13)ΔLt≤1T∑t=1T∑i=1Nφi′2v,t−φi′v,tfit+cit+git+μit.The expected lossΔLt is the distribution function of fit and git. If ΔLt is minimized and fair conditions (2) and (3) are satisfied, the expected loss will gradually decrease with time. Let
(14)Ft=εH−ΔLt,is the target number, which indicates the inequality of expected loss and waiting time between participants, and it should be minimized, where ε is a federated regularization term to constraint the equilibrium between the two targets. Therefore, the federation is aimed at the following function:
(15)Ft=1T∑t=1T∑i=1Nφi′v,tεqit+fit+cit+git+μit−φi′2v,t,and the constraint condition is
(16)∑i=1Nφ~iv,t≤bt,φ~iv,t≥0,for∀i,t,where bt means the cost of participants in the current payment budget, which is determined by their expected loss and the time they spend waiting for the reward to be fully paid, and the φ~it represents the income that participant i received from the federation in round t. Let us take the derivative of formula (15) with respect to φi′v,t, and set the first derivative equal to 0 to obtain φi′v,t:
(17)φi′v,t=12εqit+fit+cit+git+μit.Moreover, by using the second derivatives of (15) with respect to φi′t, we obtain the following result:
(18)d2φi′2v,tFt=−1<0.When the federated model runs in the roundt, the contribution of the participants to the federation is qit, and the total compensation of the participants i is φi′v,t=1/2εqit+fit+cit+git+μit. If budget bt is not enough to fully pay all participants’ compensation in round i, the federated organization will pay them in installments within a certain period. The federated organization will pay φ~iv,t in installments in round i according to the following equation:
(19)φ~iv,t=φi′v,t∑i=1Nφi′v,tbt.For the income distribution scheme in Figure1, when fit and git are equal to 0, this is a very important condition, and participant i does not invest extra cost; then, φi′v,t=εqit. After that, participant i will use the enhanced Shapley value method to evaluate its contribution based on the federation and allocate the future income. This income allocation method will first consider the participants whose expected risk is not equal to 0 and also consider the contributions of other participants to the federation. According to Figure 1 and the above theoretical analysis, we present the pseudo-code of FL incentive in Algorithm 1.Algorithm 1: FL incentive mechanism via enhanced Shapley value method.
Input:ε and bt are given by the federated system administrator. fit and git) are the expected risk of all participants and the income of all participants respectively in round t.Output:φ~1v,t,φ~2v,t,⋯,φ~Nv,t1: Initializationht⟵0; To stores the sum of all φi′v,t values.2:fori=1,2,⋯,Ndo3:ifdit>0then; dit represents the contribution scale value of participant i to the federated model.4:  Calculationcit;5:  Calculationqit;6:else7:cit=0;8:φi′v,t⟵12εqit+fit+cit+git+μit;9:ht⟵ht+φi′v,t;10:ifi=1,2,⋯,Nthen11:φ~iv,t⟵φi′v,thtbt12:fit+1=maxfit+cit−φ~iv,t,0;13:git+1=maxgit+μit−φ~iv,t,0;14:returnφ~1v,t,φ~2v,t,⋯,φ~Nv,t


## 4.3. The Enhanced Shapley Value Method

To ensure that all participants are satisfied with the federated income distribution scheme and make the distribution mechanism play an incentive role, all participants can actively contribute to the federated model. The Shapley value method considers that the contribution degree of each participant as1/n ignores the influence of other factors. Therefore, it is necessary to consider the effects of multiple factors on federal income distribution, by taking multiple influence factors as weights, and the enhanced Shapley value method is proposed to measure income distribution [28, 40].
(20)φi′v,t=φiv,t+vN×ΔPi,ΔPi=Pi−1n,i=1,2,⋯,n,(21)Pi=αi1,αi2,⋯,αinw1,w2,⋯,wnT,i=1,2,⋯,n,where φi′v,t shows the anticipated income for federated model i after improvement and under a federated game condition in the t round, vN is the revenue of the grand coalition, φiv,t is the income distribution of participant i using the Shapley value method in the t round of FL. In particular, when t=1, φi′v,t is calculated using formula 6, and ΔPi indicates the difference between the comprehensive evaluation factor Pi and the average factor 1/n introduced by participant i, and it refers to the comprehensive factor as an incentive factor in the federated distribution of incomes.
(22)ΔPi=Participantishouldbeencouraged,Pi−1n>0,Participantishouldbepunished,Pi−1n<0.Pi is the sum of the product of the measured values αi1, αi2,…, αin of the n factors and the corresponding factor weight w1, w2,…,wn, and ∑i=1nPi=1. Thus, we have
(23)∑i=1nΔPi=0,∑i∈SΔPi≥vS−∑i∈Sφiv,tvS.We solve these weights ofn influence factors αi1, αi2,…, αin with the AHP. It is easy to verify that formula (20) satisfies formulas (4) and (5).Proof.
According to formulas (4) and (23), formula (20) can be proofed as follows:
(24)∑i=1nφi′v,t=∑i=1nφiv,t+vN×ΔPi=∑i=1nφiv,t+vN∑i=1nΔPi=vN.Hence, formula (20) satisfies formula (4). In addition, according to formulas (5) and (23), formula (20) is proofed as the following:
(25)∑i∈Sφi′v,t=∑i∈Sφiv,t+vS×ΔPi=∑i∈Sφiv,t+vS∑i∈SΔPi≥vS.Hence, formula (20) satisfies formula (5).To solve the Shapley value correction factor of the federated income distribution, the AHP is used to assign the importance degree according to the scale of 1-9. The explanation of scale 1-9 is presented in Table1 [39].Table 1
The explanation of index in expert judgment matrix.
IndexExplanation1The factorsi and j are equally essential2The factori is more essential than j slightly3The factori is more essential than j obviously4The factori is more essential than j strongly5The factori is more essential than j extremely2, 4, 6, 8The index between adjacent indexes aboveReciprocalThe valueaij is obtained by factor i and factor j, and the value aji=1/aij is obtained by j comparing factor with factor iIn Table1, index indicates the intensity of importance on an absolute scale. The judgment matrix in this paper is constructed by the joint score of the expert group, and they score according to the rules in Table 1. These experts have a strong background in economics, computer, and mathematics. Hence, according to the joint score of the expert group, the judgment matrix Aaijn×n is obtained.
(26)A=a11a12⋯a1na21a22⋯a2n⋮⋮⋮⋮an1an2⋯ann,and satisfiedaij>0, aji=1/aij, and aii=1. The matrix Bbijn×n is given by normalizing, and according to each column of the matrix Aaijn×n, bij=aij/∑i,j=1naij, αi1, αi2,⋯, αin, comes from each i-th row element of matrix Bbijn×n and represents the measurement value of n factors affecting federated revenue; αij represents the j-th factor affecting the individual’s federated income, and ∑i,j=1nαij=1.
(27)B=b11b12⋯b1nb21b22⋯b2n⋮⋮⋮⋮bn1bn2⋯bnn.Sum matrixBbijn×n is according to row vector, ci=∑i,j=1nbij, and achieves the vector C.
(28)C=c1,c2,⋯,cn,and then, normalizing the vector C, wi=ci/∑i=1nci. Consequently, the weight value w of judgment matrix Aaijn×n is obtained.
(29)w=w1,w2,⋯,wnT,∑i=1nwi=1.Since the judgment matrixAaijn×n is established based on the experience and estimation of experts, some differences will appear between the judgment matrix A and the real situation, and it needs to test the consistency of the matrix Aaijn×n. If the matrix Aaijn×n is consistent, its eigenvector w=w1,w2,⋯,wnT can truly reflect the reasons that affect federated income. In this case, we can use the random consistency index (RI) [38, 39] in Table 2.
(30)λmax=∑i=1nAwinwi,CI=λmax−nn−1,CR=CIRI,where λmax is the largest eigenvalue of the matrix Aaijn×n, the CI represents public consistency index, RI is the average random consistency index, and CR is the consistency ratio. As we refer to Table 2, matrix Aaijn×n needs to be satisfied with the consistency.
(31)CR=ThematrixAaijn×nisjustwhatweneed,CIRI<0.1,ThematrixAaijn×nneedstobeadjusted,CIRI>0.1.Table 2
Random consistency index.
Order123456789RI0.000.000.520.891.111.251.351.41.45


## 5. Numerical Results

We assume that there are three banks: bank 1, bank 2, and bank 3, which are taking part in the FL model. The incomes of three banks not participating in FL arev1=150, v2=250, and v3=350. If bank 1 and bank 2 coalition can make income v1,2=400, bank 1 and bank 3 coalition can make income v1,3=800, bank 2 and bank 3 coalition can make income v2,3=700, and banks 1, 2, and 3 coalition can make income v1,2,3=1500. After the three banks unite together, the income distribution of each bank is solved.


### 5.1. Illustrating Result of the Shapley Value Method

In this subsection, the Shapley value method does not consider any factors to the federated income but distribute the income according to the average distribution method. Therefore, we can calculate the income distribution value of three banks after FL according to the relevant knowledge of cooperative game theory and Equations (6) and (7). The details are shown in Table 3.Table 3
Bank 1 federated income calculation.
S11,21,31,2,3vS1504008001500vS\10250350700vS−vS\1150150450800S1223wS1/31/61/61/3wSvS−vS\1150/3150/6450/6800/3Therefore, add the last row of Table3, and we achieve the federated income value allocated by bank 1 which is the following: φ1v,t=416.67; similarly, the federated income value allocated by bank 2 is as follows: φ2v,t=416.67, and the federated income value allocated by bank 3 is as follows: φ3v,t=666.67.


### 5.2. Illustrating Result of the Enhanced Shapley Value Method

In the FL incentive model, the income distribution of the federated bank depends not only on the marginal contribution but also on other factors that affect the total income. In this example, we mainly consider the risk factor, data quality, effort degree, and cost input. For banks with higher risk, higher input data quality, higher level of cooperation, and perhaps higher-cost investment, the weight of benefit distribution should be appropriately increased. According to the enhanced Shapley value model and the following AHP flow chart in Figure2, experts give scores to the influence factors according to Table 1, and we achieve the judgment matrix A,B1,B2,B3,B4. Here, matrix A is the weight matrix of reference layer B to target layer.Figure 2
AHP flow chart.Through the weight matrixA, the weight value of reference layer B to target layer A can be calculated. Similarly, we can obtain the weight matrix B1,B2,B3,B4 of scheme layer C to reference layer B and the corresponding weight. These weight values of influence factors and the corresponding λmax, and CR values are solved in the following sheet.Suppose three banks construct judgment matrix according to their own actual situation, and applying to formulas (20)–(31), we achieve the following conclusions:
(1)
Expert judgment matrixA−B and weight w(2)
Expert judgment matrixB1−C and weight w11(3)
Expert judgment matrixB2−C and weight w21(4)
Expert judgment matrixB3−C and weight w31(5)
Expert judgment matrixB4−C and weight w41Consequently, we let(32)w1=w11,w21,w31,w41=α11α12α13α14α21α22α23α24α31α32α33α34,and then, using formula (30), we can calculate
(33)P=w1w=0.62670.77320.37470.41610.09360.13920.05450.12600.27970.08770.57120.45790.09180.10400.25460.5496=0.46200.10620.4319,and ΔPi=Pi−1/n=0.4620, 0.1062, 0.4319T−1/3=0.1287,−0.2271, 0.0986T. According to formula (22), ΔP2<0 means that the second bank’s enthusiasm in the FL process is significantly lower than the average level of the federated banks and should be punished. Therefore, using formulas (20) and (21) can calculate the income distribution of the three banks in FL as follows:
(34)φ1′v,t=25006+1500×0.1287=609.72,φ2′v,t=25006+1500×−0.2271=76.02,φ3′v,t=40006+1500×0.0986=814.57.Through the numerical calculation of the enhanced Shapley method, we achieve the expected result which is the following:(35)∑i=13φi′v,t=609.72+76.02+814.57≈1500.After verification, it is not difficult to find that in the case of the income distribution scheme of the enhanced Shapley value method, the income of the three banks is consistent with the theory.


### 5.3. Numerical Comparison

In this subsection, we compare the results of income distribution using the Shapley value method with the results of income distribution using the enhanced Shapley value method [40] in Table 4 and simulate their values in Figure 3.Table 4
Income comparison.
ParticipantShapley value methodEnhanced Shapley value methodAverage weightIncome distributionEnhanced weightIncome distributionBank 10.3333416.670.4620609.72Bank 20.3333416.670.106276.02Bank 30.3333666.670.4319814.57Figure 3
Income histogram of federated model.Table4 indicates that the Shapley value method does not consider any factors and the corresponding weights in the income distribution; however, in the enhanced Shapley value method, we consider the factors that affect income distribution and the corresponding weights. It is not difficult to find out bank 1 has the largest weight, indicating that bank 1 is the most positive in the whole FL process, followed by the third bank, and the second bank is the worst, which should be punished.From Figure3, we can clearly see that income allocated by the Shapley value method shows that the first bank and the second bank have the same income. However, relying on the enhanced Shapley value method, it is easy to find that the second bank is the least profitable.


## 5.1. Illustrating Result of the Shapley Value Method

In this subsection, the Shapley value method does not consider any factors to the federated income but distribute the income according to the average distribution method. Therefore, we can calculate the income distribution value of three banks after FL according to the relevant knowledge of cooperative game theory and Equations (6) and (7). The details are shown in Table 3.Table 3
Bank 1 federated income calculation.
S11,21,31,2,3vS1504008001500vS\10250350700vS−vS\1150150450800S1223wS1/31/61/61/3wSvS−vS\1150/3150/6450/6800/3Therefore, add the last row of Table3, and we achieve the federated income value allocated by bank 1 which is the following: φ1v,t=416.67; similarly, the federated income value allocated by bank 2 is as follows: φ2v,t=416.67, and the federated income value allocated by bank 3 is as follows: φ3v,t=666.67.


## 5.2. Illustrating Result of the Enhanced Shapley Value Method

In the FL incentive model, the income distribution of the federated bank depends not only on the marginal contribution but also on other factors that affect the total income. In this example, we mainly consider the risk factor, data quality, effort degree, and cost input. For banks with higher risk, higher input data quality, higher level of cooperation, and perhaps higher-cost investment, the weight of benefit distribution should be appropriately increased. According to the enhanced Shapley value model and the following AHP flow chart in Figure2, experts give scores to the influence factors according to Table 1, and we achieve the judgment matrix A,B1,B2,B3,B4. Here, matrix A is the weight matrix of reference layer B to target layer.Figure 2
AHP flow chart.Through the weight matrixA, the weight value of reference layer B to target layer A can be calculated. Similarly, we can obtain the weight matrix B1,B2,B3,B4 of scheme layer C to reference layer B and the corresponding weight. These weight values of influence factors and the corresponding λmax, and CR values are solved in the following sheet.Suppose three banks construct judgment matrix according to their own actual situation, and applying to formulas (20)–(31), we achieve the following conclusions:
(1)
Expert judgment matrixA−B and weight w(2)
Expert judgment matrixB1−C and weight w11(3)
Expert judgment matrixB2−C and weight w21(4)
Expert judgment matrixB3−C and weight w31(5)
Expert judgment matrixB4−C and weight w41Consequently, we let(32)w1=w11,w21,w31,w41=α11α12α13α14α21α22α23α24α31α32α33α34,and then, using formula (30), we can calculate
(33)P=w1w=0.62670.77320.37470.41610.09360.13920.05450.12600.27970.08770.57120.45790.09180.10400.25460.5496=0.46200.10620.4319,and ΔPi=Pi−1/n=0.4620, 0.1062, 0.4319T−1/3=0.1287,−0.2271, 0.0986T. According to formula (22), ΔP2<0 means that the second bank’s enthusiasm in the FL process is significantly lower than the average level of the federated banks and should be punished. Therefore, using formulas (20) and (21) can calculate the income distribution of the three banks in FL as follows:
(34)φ1′v,t=25006+1500×0.1287=609.72,φ2′v,t=25006+1500×−0.2271=76.02,φ3′v,t=40006+1500×0.0986=814.57.Through the numerical calculation of the enhanced Shapley method, we achieve the expected result which is the following:(35)∑i=13φi′v,t=609.72+76.02+814.57≈1500.After verification, it is not difficult to find that in the case of the income distribution scheme of the enhanced Shapley value method, the income of the three banks is consistent with the theory.


## 5.3. Numerical Comparison

In this subsection, we compare the results of income distribution using the Shapley value method with the results of income distribution using the enhanced Shapley value method [40] in Table 4 and simulate their values in Figure 3.Table 4
Income comparison.
ParticipantShapley value methodEnhanced Shapley value methodAverage weightIncome distributionEnhanced weightIncome distributionBank 10.3333416.670.4620609.72Bank 20.3333416.670.106276.02Bank 30.3333666.670.4319814.57Figure 3
Income histogram of federated model.Table4 indicates that the Shapley value method does not consider any factors and the corresponding weights in the income distribution; however, in the enhanced Shapley value method, we consider the factors that affect income distribution and the corresponding weights. It is not difficult to find out bank 1 has the largest weight, indicating that bank 1 is the most positive in the whole FL process, followed by the third bank, and the second bank is the worst, which should be punished.From Figure3, we can clearly see that income allocated by the Shapley value method shows that the first bank and the second bank have the same income. However, relying on the enhanced Shapley value method, it is easy to find that the second bank is the least profitable.


## 6. Conclusion and Future Work

Federated learning (FL) is a distributed machine learning framework, and how to design an incentive mechanism for federated participants to sustain participate in model training is very important to train an accurate federated model. In this paper, we propose an incentive mechanism using the enhanced Shapley value method, and this incentive mechanism considers four factors that affect the federated income distribution, which can better reflect the fairness of income distribution. Further, the analytic hierarchy process is used to calculate the corresponding weight of four factors. Numerical results verify that the income distribution of all participants can better reflect multiple factor contribution when using the incentive mechanism with the enhanced Shapley value method, which gets a better incentive for federated participants.In the next step, we will apply the proposed incentive mechanism to solve the problem of participants’ income distribution in FL train models. This incentive mechanism can be applied in WeBank, such as financial institutions, insurance companies, and enterprises, for reducing loan risks and improving economic earning significantly in practice.



---

*Source: 9690657-2022-01-28.xml*