# A Fairness-Aware Peer-to-Peer Decentralized Learning Framework with Heterogeneous Devices
## 4 Fairness-Aware Decentralized Learning
### Our Fairness-Aware Approach  
We present our approach to solve the decentralized learning optimization problem, including the consideration of learning fairness, the design rationale, and our workflow. In the decentralized learning, the local learner trains the learning model with its dataset. The learning objective is determined by its local preference. To obtain the local learning model from other nodes, the local learner needs to find a learning pair by a peer-to-peer manner. In this regard, the optimization goal of each local learner can be considered as:  
\[\mathbf{w}_{i}^{t+1}=\mathit{arg~{}min}F_{i}(\mathit{Agg}(\mathbf{w}_{i}^{t},\mathbf{w}_{j}^{t}),D_{i}),\forall\;j\in N. \tag{3}\]  
Here, \(\mathit{Agg}(\cdot)\) and \(t\) are the aggregation scheme and the learning pair (iteration), respectively. Moreover, the model aggregation, in most previous works, adopts the weight averaging-based algorithms [16]. Such aggregation schemes could degrade the learning performance, if the local datasets are significantly statistical heterogeneous. For example, a number of local nodes with non-i.i.d. data distribution and the extreme massive local dataset might dominate the training process, resulting that the learning model benefits partial participants. To address such a learning fairness issue, we design an adaptive local aggregation scheme which satisfies the following requirements: (i) the proposed aggregation scheme should be adaptive and dynamic based on the learning performance; (ii) the training participants have different local learning objectives with their own preferences; (iii) the proposed aggregation scheme should be lightweight, not bringing heavy computing load to the local learner. In detail, our aggregation scheme is as follows:  
\[P_{i}^{t}=\frac{\mathit{pef}_{\mathbf{w}_{i}D_{i}}^{t}-\mathit{pef}_{ \mathbf{w}_{i}D_{i}}^{t}}{2*\mathit{pef}_{\mathit{max}}}+C, \tag{4}\]  
where \(P_{i}^{t}\) is the weight of learning model \(\mathbf{w}_{i}\) in the learning pair \(t\), \(\mathit{pef}_{\mathbf{w}_{i}D_{i}}^{t}\) is the learning performance of learning model \(\mathbf{w}_{i}\) over the dataset \(D_{i}\), and \(\mathit{pef}_{\mathit{max}}\) is the best learning performance. Note that we have multiple performance indicators for the learning performance _pef_ in Equation (4) (accuracy, loss value, etc.). As demonstrated in Equation (4), the proposed aggregation scheme allows each local learner to determine the aggregation weights with local preferences (\(D_{i}\)). To determine the weight of learning model \(\mathbf{w}_{j}\) on the learner \(i\), we have  
\[P_{i,j}^{t}=1-P_{i}^{t}. \tag{5}\]  
We show the details of our approach in Algorithm 1. Particularly, each local client finds a learning pair with a peer-to-peer manner iteratively, where the local learning objective is determined by the device owner. To generate a local update, all clients adopt a gradient descent-based mechanism [33]. For Line 4 to Line 7, it shows that (i) a client requests the local learning model from its learning pair; (ii) the client evaluates the learning performance of received learning model; and (iii) it adaptively assigns the weight of the received model based on its learning performance (using Equations (4) and (5)). The complexity of our proposed algorithm on the client side is controlled by the number of learning pairs subject to available resources. Note that we assume each local client is capable of finding a learning pair in the training process by using technologies such as 5G, IoT search engine [34].