FedToken: Tokenized Incentives for Data Contribution in Federated Learning
Abstract
Incentives that compensate for the involved costs in the decentralized training of a
Federated Learning (FL) model act as a key stimulus for clients’ long-term participation.
However, it is challenging to convince clients for quality participation in FL due to the absence of: (i) full information on the client’s data quality and properties; (ii) the value of client’s data contributions; and (iii) the trusted mechanism for monetary incentive offers.
This often leads to poor efﬁciency in training and communication.
While several works focus on strategic incentive designs and client selection to overcome this problem, there is a major knowledge gap in terms of an overall design tailored to the foreseen digital economy, including Web 3.0, while simultaneously meeting the learning objectives.
To address this gap, we propose a contribution-based tokenized incentive scheme, namely FedToken , backed by blockchain technology that ensures fair allocation of tokens amongst the clients that corresponds to the valuation of their data during model training.
Leveraging the engineered Shapley-based scheme, we ﬁrst approximate the contribution of local models during model aggregation, then strategically schedule clients lowering the communication rounds for convergence and anchor ways to allocate affordable tokens under a constrained monetary budget.
Extensive simulations demonstrate the efﬁcacy of our proposed method.
Introduction 1.1 Background and Motivation
The next generation of internet architecture is said to be backed by Web 3.0, a step further toward a fully decentralized online ecosystem powered by blockchain technologies [1–4].
This novel connectivity paradigm, though still in its infancy, opens up unique ways to monetize decentralized participation and allows clients of heterogeneous proﬁles to engage in the secure exchange of data, assets and trust.
Recent paradigm of privacy-preserving decentralized training of machine learning models, such as Federated Learning (FL) [5–7], is another complementary enabler that is already offering various applications on top of the existing network architecture via distributed computing collaboration on private, untapped data.
Therein, it is important to facilitate appropriate participation for distributed training of FL models to support machine learning applications of interest, for instance, in healthcare, automation, logistics and much more [6, 8, 9].
An efﬁcient incentive design is thus a fundamental stimulus for participation in a collaborative operation over a decentralized network infrastructure.
On that pretext, we begin with a natural question on incentive design to enable FL: How to ensure quality participation with contribution-based incentives that meet the objective of communication efﬁciency simultaneously?
This leads to investigating: (i) ways to quantify the expected contribution of clients’ shared model parameters in improving the model performance during aggregation, given constraints on the offered monetary budget; and (ii) transparent distribution of incentives as per the contribution, both, in terms of participation and the contribution made in the FL training.
Related works (cf. Section-2) do not cover these issues jointly.
In this work, we propose an overall design tailored to the foreseen digital economy that simultaneously meets the learning objectives in terms of training as well as communication efﬁciency.
We propose a contribution-based tokenized incentive scheme backed by blockchain technology.
It ensures a fair allocation of tokens amongst the clients as per the value of contribution with their shared local model parameters during model training.
Our main contributions are as follows:
• Tokenized Incentives: We develop tokenized incentives FedToken 1 for clients to train the FL learning models under budget constraints, reﬂected with quota , at the aggregator.
We propose a contribution accounting mechanism based on blockchain.
The token distribution algorithm is designed to attribute the value of participation in the model aggregation period.
• Robust System Design and Improved Performance :
The token allocation scheme accounts for the objective of communication efﬁciency, achieved as a direct consequence of the strategic selection of client’s updates during model aggregation in FL.
The rewards for participation are shared fairly: we use temporal discounting amongst the clients that transmitted low-quality model updates.
This discourages free-riders .
We incentivize quality updates in a way that is robust to any choice of the local learning algorithm.
• Implementation Features and Findings: The activities regarding the exchange and allocation of tokens are immutably recorded into the Blockchain.
The contributors can query the relevant information, such as account balance, receiving tokens, or missing contributions.
This is the trusted platform that ensures quality participation in federated learning.
Related Work
Incentive Mechanism for FL.
Incentive designs to enable efﬁcient and effective participation of clients in training FL models have been a key focus in recent works [10–12, 2].
In line to this, one fundamental research question on incentive design for FL is the value of data contribution .
Most incentive mechanism design for FL was investigated dominantly in the wireless regime so as to enable participation while optimizing network resources [11, 10, 13, 12], rather than accounting for the impact of individual data contributions.
The authors in [14] showed a contribution-index based on an approximation to the Shapley value [15, 16] to evaluate data value in FL.
The authors in [17] presented a novel multidimensional incentive framework for FL.
The authors in [18] proposed a game-based incentive mechanism for an FL platform that combines distributed deep learning and crowdsensing technique for big data analytics on mobile clients.
In [19], the authors propose a novel incentive governor for an efﬁcient FL in highly heterogeneous and dynamic scenarios considering contribution and reputation.
However, these works primarily have two issues.
First , the scoring mechanism is too subjective and lacks quality evaluation schemes.
Second , each participant only has a score and can be easily affected by malicious raters.
Therefore, in the absence of a trusted mechanism for recording the offered incentives, the solution is not viable for practical usage.
Furthermore, despite a plethora of incentive designs to enable FL, it has not been formalized well in the literature, what exactly are these incentives, anyway ?
Incentive Mechanism in Blockchained FL.
Feasibility study on the integration of FL with blockchain is explored in several literature [20–23].
One key beneﬁt is blockchain provides a trustworthy platform to realize tokenization , which is a process of creating and managing tokens by recording the veriﬁable title transfer from one user to another.
Tokens , in principle, then could quantify offered incentives in FL.
Native tokens on a blockchain (e.g., BTC[24] on Bitcoin, ETC on Ethereum [25]) can be used to formulate a value system where the tokens represent monetary value or physical assets offered for quality participation in FL training.
However, most of the mentioned works are about latency analysis and scalability issues rather than contribution assessment and token allocation for improving learning objectives in FL.
The authors in [26] present a blockchain-based reputation system to train FL models.
In [27], the (mining) rewards are allocated as per the local data size and compute dominance, which doesn’t reﬂect a practical scenario.
Moreover, budget constraints are overlooked, let alone the learning performance.
In line with this, the authors in [28] provide a comprehensive survey on blockchain-based FL, where the use of blockchain in FL is reviewed for security breaches and honest participation of clients.
The work [23], which is close to our research, introduces a tokenized incentive mechanism where tokens are used as a means of paying for the services that provide participants and the training infrastructure.
However, the approach of computing individual contributions is different and for a separate context.
In addition, it is intuitively to argue that participation only cannot/shouldn’t ensure tokens.
We address this and consider contributions in participation and allocate tokens without compromising the learning objectives.
We show that irrespective of participation frequency, only quality updates are considered for the tokenized incentives, and strategic allocation of tokens jointly improves training and communication efﬁciency in FL.
Overall System Design
Consider a set of clients [ N ] = { 1 , 2 , . .
. , N } and the training examples held by each client n ∈ [ N ] as a collection of D n input-output pairs { x i , y i } D n i
=1 , where, respectively, x i ∈ R
d is an input and y i ∈ R .
Each client n exchanges updates on computations of their local data set D n with the shared edge aggregator to collaboratively train a learning model.
In FL, this translates to ﬁnding an optimal global model parameter w ∈ R d that minimizes the empirical risk on all of the distributed training data samples, i.e., the clients collaborate to solve the following optimization problem
[Equation]
where f n ( w ) captures the cost of prediction with the model parameter w ; λ is a regularizer; and g ( w ) is a regularization function to control model complexity and offer generalization .
In practice, the training examples ( x i , y i ) are drawn from D n , and as such, the total available training samples is D = (cid:80)
Nn =1 D n .
Then, we formally deﬁne f n ( w ) =
E ( x i ,y i ) ∼D n [ l ( w ; ( x i , y i ))]
as the expected loss. Here, l ( w ; ( x i , y i )) is the measuring loss function for the prediction [7, 5] over data samples ( x i , y i ) , ∀ i ∈ D n .
Assumption 1 ( General Assumptions ) .
The loss function f n : R d → R is twice continuously differentiable, L − smooth and µ − strongly convex, that means, we have
[Equation]
where I ∈ R
d × d is the identity matrix and ∇ 2 f n ( w ) is the Hessian of f n in w , respectively.
Following Assumption 1, we solve the learning problem (1) in its dual form as a particular case of Fenchel duality [29, 10].
The separation of the global problem (1) as primal-dual updates, on the one hand, allows clients to control their local computations in improving the quality of local updates and, on the other hand, facilitates the aggregator to evaluate contributions made by the clients to perform efﬁcient incentives allocation during model training.
Then, (1) in its dual form considering the availability of overall D data samples, using α ∈ R D as the dual variable mapping to the primal candidate vector and φ ( α ) = 1 λD Xα , is deﬁned as
[Equation]
where f ∗ n and g ∗ are the convex conjugates of f n and g , respectively, and X ∈ R d × D is a matrix with each column representing data points.
Then, we get the mapping between the optimal value of dual variable α ∗ in (2), and the optimal solution of (1) following the ﬁrst-order optimality conditions [29] as w ( α ∗ ) =
∇ g ∗ ( φ ( α ∗ )) .
Using a shorthand notation for φ ∈ R d as φ ( α ) , in the following, we outline the primal-dual solution abusing distributed computation of clients in a systematic manner.
3.1 Primal-dual solution approach
We resort solving (2) in a distributed manner, as shown in [29–31], with the iterative exchange of local dual variables α [ n ] , (cid:37) [ n ] obtained at each client n ∈ [ N ] .
Note that both α [ n ] , (cid:37) [ n ] are vectors in R D with non-zero elements for the residing local data points, i.e., for instance, ( α [ n ] ) i
= α i if i ∈ D n and 0 otherwise.
The selected clients iterates over their local training examples to output α [ n ] , (cid:37) [ n ] .
Formally, in one single global communication round t ∈ { 1 , 2 , . .
. , τ } , i.e., the interaction between the edge aggregator and the clients, each selected client n solves the following dual problem in the distributed setting:
[Equation]
where the weight vector (cid:37) [ n ] ∈ R D has elements zero for the unavailable data points and reports value only for the coordinates corresponding to local variables α [ n ] , and 0 otherwise; F n ( (cid:37) [ n ] ; φ, α [ n ] )
= − 1 N g ∗ ( φ ( α )) −(cid:104) 1 D X T [ n ] ∇ g ∗ ( φ ( α )) , (cid:37) [ n ] (cid:105)− λ 2 (cid:107) 1 λD X [ n ] (cid:37) [ n ] (cid:107) 2 − 1 D (cid:80) i ∈D n f ∗ i ( − ( α [ n ] ) i − ( (cid:37) [ n ] ) i ) is a corresponding quadratic approximation of F at ( α [ n ] + (cid:37) [ n ] ) with X [ n ] as a matrix with i − th column having data points for i ∈ D n , i.e., ( X [ n ] ) i = ( X ) i , and zero-padded otherwise.
As mentioned, the selected devices invest local computations in running stochastic gradient descent (SGD) on their local training samples and later share the output, i.e., the obtained local parameters ∆ φ t [ n ] := 1 λD X [ n ] (cid:37) t [ n ] , with the edge aggregator for model aggregation.
Meanwhile, the local variable gets updated in each global communication as α t +1[ n ] = α t [ n ] + ν(cid:37) t [ n ] , where ν ∈ [0 , 1] is the tuneable local aggregation parameter often set between 1 N and 1 .
3.2 Contribution-based Aggregation
Once getting the local updates from N clients, a plain model aggregation scheme is Federated
Averaging (FedAvg) [5], i.e., we compute the average of received local updates as
[Equation]
and pass φ t +1 back to the [ M ] ⊆
[ N ] random clients to compute (3), and iteratively as such, thereafter.
We, however, resort to the engineered, cost-efﬁcient Shapley-based scheme [32, 31] that allows approximation of the contribution of local models during model aggregation to lower the overall communication overhead and design tokenize incentives based on that.
The edge aggregator performs δ random permutations over the received dual variables, where the average marginal contribution is evaluated as
[Equation]
Here, p ∈ P is the permutation of clients updates, i.e., ∆ φ t [ n ] , ∀ n ∈ [ M ] , with a probability of 1 | M | ! ;
V ( · ) is monotone quantifying performance score evaluated on the test dataset with the permuted subset of client updates, which translates to the improvement in global model accuracy and is measured by (cid:15) ; S n,p is a set of preceding clients before n for a permutation p selected for model aggregation – then, all possible orders of clients contribute to the average contribution (5).
Furthermore, we assume the availability of test dataset at the aggregation server; having it as such is a common assumption in FL [6, 33], particularly, for achieving model robustness and a fair level of security guarantees.
Then, formally, for a test dataset ( x i , y i ) ∼ D , V ( D ) =
V ref − 1 |D| (cid:80) |D| i =1 l ( w ( α ); ( x i , y i )) such that F ( α ∗ ) ≤ l (; ) ≤ V ref with E [ F ( α ) − F ( α ∗ )] < (cid:15) .
The method of evaluating the marginal contribution of the local models during the aggregation period in (5) primarily ignores the constrained monetary budget at the aggregator.
Said the monetary budget can be quantiﬁed as tokens (see Deﬁnition 1), then, in principle, response to all M queries made by the aggregator, i.e., the local variables received from M participating clients, cannot be incentivized with tokens except with a compensation for their participation as participation reward.
Next, while keeping the learning objectives intact, this raises the following two research questions : (i) how many updates can the aggregator afford with the given budget , and (ii) how to fairly allocate the available budget given only a subset of received local updates contribute in improving the model performance?
This also challenges ﬁnding comparable contributions, discouraging free-riders, and handling possible attacks during model aggregation.
Therein, the tokens allocation scheme has to be justiﬁed as per the accounted marginal contribution of the local updates and the available monetary constraints at the edge aggregator.
Once done, the records of the contribution accounting mechanism are kept on the Blockchain, hence, allowing contributors to make queries about the relevant information during FL training. Select ; 4: for all do 5: Get :
∆ φ m ; 6: end SetCounter: c 0 ; 8: while c < δ do 9:
{+Set : v c 
+}
= V (
; 10: for all m ∈ [ M ] do 11: if | V ( φ c ) − v cm − 1 | < (cid:15) then 12:
Update: v cm = v cm − 1 ; 13: else 14:
Set: v cm ← V ( { p c [1] , . . .
, p c [ m ] } ); 15: end if 16:
Update: u p c [ m ] ← [ c − 1 c ] u p c − 1 [ m ] + 1 c
[ v cm − v cm − 1 ] ; 17: end for 18:
UpdateCounter : c = c + 1 ; 19: end while 20:
Sort[M] : Sort clients in the descending order of u p c [ m ] ; 21:
Aggr[ Q ] : Select |Q clients updates for aggregation; 22: for all q ∈ [ Q ] do 23:
TokenAllocation ; 24: end for 25:
Update :
; 26: end for
We formalize this with the following deﬁnition on the constrained monetary budget at the aggregator.
Deﬁnition 1 ( Quota ) .
We deﬁne quota [ Q ] ⊆
[ M ] as the affordable number of enrolled model updates with the ﬁxed monetary budget B that satisﬁes the pre-deﬁned global model accuracy 0 ≤ (cid:15) ≤ 1 (i.e., E
[ F ( α ) − F ( α ∗ )]
< (cid:15) ) upon the exchange of dual updates α .
Then, given B that translates into tokens, we allocate tokens as per the proportion of contribution amongst the chosen [ Q ] .
For simplicity, we assume B equals to the number possible tokens.
Note that the dynamic adjustment of |Q| in relation to changing the value of available budget B is left as a future work; here, we focus our analysis on a ﬁxed B and constrained quota.
Then, with
Deﬁnition 1, we have the following proposition.
Proposition 1.
For a ﬁxed B , assuming the absence of any malicious updates, a larger size of [ Q ] ⊆
[ M ] undervalues the client’s contribution as it leads to a lower average marginal valuation of the client local updates, and hence, affects the number of allocated tokens.
Proof Sketch.
The proof follows the properties of standard Shapley value [34] derived from the concept of cooperative games while evaluating the marginal contributions in (5).
Take a speciﬁc permutation instance p ∈ P , where the score (or the payoffs) with predecessor set of client n , i.e.,
S n,p = { m ∈ [ M ] | p ( m ) < p ( n ) } is V ( S n,p ) .
Then, following Linearity and Symmetry properties of
V ( S n,p ) , we can generalize V ( S n,p ∪ { n } ) ≤ V ( S n,p ∪ { n } ∪ { m } ) , ∀ n, m ∈
[ Q ] .
That means the average marginal contribution gets lowered, whereas the equality holds only if the local updates from { m } are invaluable (or malicious).
In addition, with the increase in the rounds of permutation δ and ﬁxed an affordable number of available quotas |Q| , the range of V ( Q ) changes; hence, affecting the quantity of allocated tokens.
Given the contributions are only ranked in order, Θ(log |Q| ) , is the best proportional fairness, and enjoys a permutation complexity of O ( M log M ) [32].
The details of the overall operation are provided as a pseudo-code in Algorithm 1.
[Figure]
of |Q| / | M | Ratio.
[Figure]
With the primal-dual formalization of the global learning problem, the participating clients can employ any arbitrary algorithm to solve the local learning problem and trade the output dual variables with the edge aggregator.
Following the contribution-based aggregation scheme, the edge aggregator efﬁciently allocates tokens as per its available budget opting for the following simple token distribution schemes: (i) Proportional Fair (PF) allocation, where the tokens are distributed as per the proportion of contribution in improving the global model performance, (ii) Equal Pay (EP), where all participating clients are allocated equal shares of available tokens.
Note that the proposed incentive design offers set-aside tokens for the initial participation.
For t ∈ { 2 , 3 , . . .
τ } , to eliminate the free-riders problem, we multiply the initial allocated slack tokens with a discount factor ζ t < 1 ; hence, disincentivizing poor local updates.
Performance Evaluation
Dataset and Model.
To validate the effectiveness of our FL design with tokenomic mechanism, we conduct experiments with well-known CIFAR-10 datasets [35], which covers 60000 data samples, having 32 × 32 colored images in 10 classes equally.
There are 50000 training images and 10000 test images.
We leverage a convolutional neural network (CNN) model that is maintained in a distributed manner through blockchain technology.
The designed network model possesses hidden layers for feature extraction and fully connected layers for classiﬁcation.
Implementation details.
We follow [36] to replicate the non-i.i.d.
characteristics of dataset across clients.
We set N = 100 clients on a single server with 26 core Intel Xeon 2.6 GHz, 256 GB RAM, 4 TB, Nvidia V100 GPU, Ubuntu OS and assume that the clients and server are synchronized, and evaluate the model performance by considering the impact of available quotas as a selection strategy during the model aggregation.
For training, each client model is trained for 10 epochs by SGD with a momentum 0.9, batch size of 10 , and the learning rate 0 .
Unless speciﬁed, we set the value of
M = [0 . 1 × N ] , which exhibits a good balance between computational efﬁciency and convergence rate, as demonstrated experimentally in [5].
The default value of discounting factor ζ t is set to 0.7.
Baseline.
The overall system design considers strategic model aggregation for improving the learning objective and distribution of tokens amongst clients for training FL.
In particular, the FedToken is built up based on the contribution-based aggregation to enhance the quality of updates from FL clients while reducing the communication overhead.
Therefore, we evaluate the performance of our proposed FedToken with comparable and competitive client selection, and aggregation schemes
FedAvg [5], and FedFomo [36], and further present an analysis of two tokenized incentive schemes, as PF and EP.
We tokenized the contributions and reward the participating clients based on the ﬁxed monetary budget B and corresponding affordable quota |Q| per training round.
Results .
Fig.
1a shows the impact of available quotas |Q| on the training efﬁciency.
We see the contribution-based model aggregation strategy strengthens the performance of FedToken , as compared to FedAvg, and FedFomo.
Moreover, given a low availability of |Q| , i.e., the monetary constraints or the market for participation, the signiﬁcance of including quality local updates from
FL clients is prominent and is observed to improve the test accuracy up to 12 .
5% of FL.
In speciﬁc, when the ratio of |Q| / | M | < 0 . 8 , the FedToken achieves better accuracy and converges faster than the baselines.
Next, in Fig.1b, we provide an evaluation of tokenized incentives per round, where we observe clients are encouraged to join and contribute quality local updates in the training process for higher token offers – leading to competitiveness in individual contributions.
The ﬁgure compares the amount of tokens received by a client under different scenarios (EP and PF).
First, to discourage free-riders and stale updates, as participation tokens are reserved and granted, a discount factor ζ t = 0 .
7 , ∀ t , is multiplied with the allocated tokens.
Second, FedToken can effectively detect data poisoning attacks resulting in undesirable local updates, thereby not allocating any participation tokens and saving the token budget for the rest.
Fig.
1c demonstrates the impact of random permutation in evaluating the value of local updates in model aggregation.
We observe a fundamental limit on imposing larger local computations versus the number of permutations at the aggregator for improving model performance.
This means the aggregator can compensate the costs of local computation with added δ , revealing the scope for incentive management.
In Fig.
1d, we analyze the communication overhead of FedToken as compared with the discussed baselines.
The proposed approach is communication-efﬁcient and offers better convergence with a low overhead of approx.
76% and 29 .
47% on average in comparison with FedAvg and FedFomo, respectively, due to the strategic selection of local updates as the contribution abiding available quotas.
Finally, Fig.
1e evaluates the impact of budget constraint B = 1000 tokens with the affordable quota |Q| on the learning performance.
To begin with, each participant is allocated with a token in their balance and later discounted as per their contribution if not selected during aggregation.
We observe a relation between test accuracy and the rounds of communication to exhaust the available tokens for different values of quotas.
In principle, strategic selection of clients update allows optimal use of available tokens, as ﬁrst argued in Fig.
1a.
This further opens up several research opportunities, for instance, to design a collaborative data trading market relating tokens and purchasing quotas.
Conclusion
In this work, we proposed tokenized incentives for clients contributing quality updates to train an FL model in a communication-efﬁcient manner.
We developed a holistic approach where token allocation under budget constraints, formalized as quota, is based on the value of contribution during the model aggregation phase, which later favors quality participation by virtue of an efﬁcient selection of local model parameters.
The proposed tokenized incentive design enables robust integration of clients with heterogeneous proﬁles for collaborative FL training while discouraging poor updates and attacks in the foreseen decentralized network architecture as Web 3.0.
Finally, we evaluated and analyzed the performance of the proposed approach via extensive simulations.
Acknowledgements and Disclosure of Funding.
This work was supported by the Villum Investigator Grant “WATER” from the Velux Foundation,
Denmark.