# QuoTa: An Online Quality-Aware Incentive Mechanism for Fast Federated Learning
## 4 Design of QuoTa
### Model Quality Detection  
Data owners may use unclaimed data resources for model training and even make a malicious attack, thus affecting global model aggregation. However, due to privacy concerns, the server cannot access each data owner's original data and thus fails to verify whether he uses the claimed training data. Instead, only the local model will be sent to the server after each FL round ends. Thus, we designed a model quality detection method for quantifying the quality of local models uploaded by data owners. After the \(t\)-th round of communication ends, the server will use the local test set to perform accuracy detection for the local model submitted by data owner \(i\) and use Formula (7) to update the quality detection value for each data owner. \[v_{i}^{t}=[(GA^{t}-LA_{i}^{t})-e_{1}^{t}]-[(GA^{t}-GA_{-i}^{t})-\epsilon_{2}^{t }], \tag{7}\]  
where \(GA^{t}-LA_{i}^{t}\) measures the difference between the data owner \(i\)'s local model accuracy \(LA_{i}^{t}\) and the global model accuracy \(GA^{t}\). The smaller above difference reflects their larger contribution for FL aggregation. In addition to that, \(GA^{t}-GA_{-i}^{t}\) means the difference between two global model accuracy values with or without these data owner \(i\), which quantifies their marginal distribution. Conversely, the smaller difference means their smaller marginal distribution. Thus, the former difference minus the latter one is used to quantify the whole difference in terms of both direct and marginal distribution. Moreover, we compare the above two differences with two well-designed threshold values, i.e., \(\epsilon_{1}^{t}\) and \(\epsilon_{2}^{t}\), respectively. Specifically, the thresholds \(\epsilon_{1}^{t}\) and \(\epsilon_{2}^{t}\) are determined by Formulas (8) and (9), which take the average of all the differences. \[\epsilon_{1}^{t}=\frac{\sum_{i\in S^{t}}(GA^{t}-LA_{i}^{t})}{|S^{t}|} \tag{8}\]  
\[\epsilon_{2}^{t}=\frac{\sum_{i\in S^{t}}(GA^{t}-GA_{-i}^{t})}{|S^{t}|} \tag{9}\]  
Consequently, the above quality detection value can be used for quantifying each data owner's direct quality at current round \(t\). When \(v_{i}^{t}\) is positive, it means this data owner performs poorly at round \(t\), and we subtract a penalty term from the previous comprehensive reputation value \(Rep_{i}^{t-1}\) in Formula (10). On the contrary, when \(v_{i}^{t}\) is negative, it means they perform better, and thus we exploit a reward item to be added to \(Rep_{i}^{t-1}\) in Formula (11). \[rep_{i}^{t}=Rep_{i}^{t-1}-v_{i}^{t} \tag{10}\]  
\[rep_{i}^{t}=Rep_{i}^{t-1}+\frac{(-v_{i}^{t})}{2}, \tag{11}\]  
where \(Rep_{i}^{t-1}\) represents their comprehensive reputation value of the data owner \(i\) in the past \(t-1\) rounds of communication, and \(rep_{i}^{t}\) means the direct reputation value at current \(t\)-th round. Specifically, \(Rep_{i}^{t}\) is updated according to Formula (12) (Here, we exploit an exponential moving average of data owner \(i\)'s reputation value to update theircomprehensive reputation value considering that the reputation value closer to current round reflects their real model quality better). \[Rep_{i}^{t}= \begin{cases}Re{p_{i}^{t-1}},&\text{if }x_{i}^{t}=0\\ \lambda*Rep_{i}^{t-1}+(1-\lambda)*rep_{i}^{t},&\text{if }x_{i}^{t}=1\end{cases}  \tag{12}\]  
where \(Rep_{i}^{t}\) is the exponential moving average about \(rep_{i}^{t}\) within \([1,t]\), and \(\lambda\) is a decay factor. When the data owner is selected at this round, their comprehensive reputation will be updated. Otherwise, it remains unchanged. When aggregating local models into global models, we compute the aggregated weight \(a_{i}^{t}\) of each data owner by Formula (13):  
\[a_{i}^{t}=\frac{LA_{i}^{t}}{\sum_{i\in S^{t}}(LA_{i}^{t})}, \tag{13}\]  
where \(LA_{i}^{t}\) is their local model accuracy at round \(t\). Compared with FedAvg, our approach is more effective at down-weighting malicious data owners and giving higher weight to benign data owners. In this way, the global model obtained by our aggregation strategy performs better than FedAvg.