# D2D-Assisted Adaptive Federated Learning in Energy-Constrained Edge Computing
## 4 Partite Graph-Based FL Scheme with Adaptive D2D Pairing and Aperiodic Local Iterations
Based on the above considerations, in this section, we will explore the gain of dataset correlation between client servers in model aggregation process, and leverage graph learning and deep reinforcement learning (DRL) to propose a D2D-assisted FL scheme. Let \(g_{n,m,k}\) denote the model training gain brought about by the aggregation of client server \(n\) and relay server \(m\) due to their dataset correlation in the \(k\)-th round of the FL iteration. Since there exists a complex and implicit relationship between the datasets correlation and the model gains, it is difficult to give an explicit formula for \(g_{n,m,k}\). Thus, we use a graph to present this relationship, which can be denoted as \(G(\mathcal{V},\mathcal{E},\mathcal{X})\). Here, \(\mathcal{V}\) is the set of vertices, composed of FL client servers, and \(\mathcal{E}\) is the set of edges between the vertices. The edge between any two vertices only exists when their corresponding servers \(n\) and \(n^{\prime}\) satisfy

\[L(\omega_{n,k}^{\prime}|\mathcal{D}_{n,k},\tau_{n,k})\leq L(\omega_{n,k}| \mathcal{D}_{n,k},\tau_{n,k})+\bigtriangledown L(\omega_{n,k}|\mathcal{D}_{n, k},\tau_{n,k})(\omega_{n,k}^{\prime}-\omega_{n,k})+\frac{\zeta}{2}||\omega_{n,k}^{ \prime}-\omega_{n,k}||^{2}\enspace. \tag{13}\]4

In graph \(G(\mathcal{V},\mathcal{E},\mathcal{X})\), \(\mathcal{X}=\{\mathcal{X}_{n}\}\) is the attribute of vertex \(n\), such as the collection place and collection frequency of the dataset used by the server corresponding to \(n\), and the server's remain energy budget. We use \(h_{n,k}\) to denote the features embedded in vertex \(n\) in round \(k\) of the FL, which can be calculated as

\[L(\omega_{n,k}^{\prime}|\mathcal{D}_{n,k},\tau_{n,k})\leq L(\omega_{n,k}| \mathcal{D}_{n,k},\tau_{n,k})+\bigtriangledown L(\omega_{n,k}|\mathcal{D}_{n, k},\tau_{n,k})(\omega_{n,k}^{\prime}-\omega_{n,k})+\frac{\zeta}{2}||\omega_{n,k}^{ \prime}-\omega_{n,k}||^{2}\enspace. \tag{13}\]5

where \(h^{(0)}_{n,k}=\mathcal{X}_{n}\) and \(\phi^{(l)}\) and \(B^{(l)}\) are parameters of the \(l\)-th layer of the graph neural network and \(\mathbb{N}(n)\) is the neighbor vertices of vertex \(n\). Based on the formulated graph, the model training gain \(g_{n,m,k}\) is represented as

\[L(\omega_{n,k}^{\prime}|\mathcal{D}_{n,k},\tau_{n,k})\leq L(\omega_{n,k}| \mathcal{D}_{n,k},\tau_{n,k})+\bigtriangledown L(\omega_{n,k}|\mathcal{D}_{n, k},\tau_{n,k})(\omega_{n,k}^{\prime}-\omega_{n,k})+\frac{\zeta}{2}||\omega_{n,k}^{ \prime}-\omega_{n,k}||^{2}\enspace. \tag{13}\]6

In (28), \(\mathcal{P}_{n,k}\) is the negative sampling distribution of server \(n\), and

\[L(\omega_{n,k}^{\prime}|\mathcal{D}_{n,k},\tau_{n,k})\leq L(\omega_{n,k}| \mathcal{D}_{n,k},\tau_{n,k})+\bigtriangledown L(\omega_{n,k}|\mathcal{D}_{n, k},\tau_{n,k})(\omega_{n,k}^{\prime}-\omega_{n,k})+\frac{\zeta}{2}||\omega_{n,k}^{ \prime}-\omega_{n,k}||^{2}\enspace. \tag{13}\]7

Here, we leverage the cosine embedding loss function to calculate the embedding similarity between server \(n\) and \(m\), as well as the negative sample embedding dissimilarity between the server pairs [33]. In the cosine function, \(\|h_{n,k}\|\) and \(\|h_{m,k}\|\) are the norms of vectors \(h_{n,k}\) and \(h_{m,k}\), respectively, where

\[L(\omega_{n,k}^{\prime}|\mathcal{D}_{n,k},\tau_{n,k})\leq L(\omega_{n,k}| \mathcal{D}_{n,k},\tau_{n,k})+\bigtriangledown L(\omega_{n,k}|\mathcal{D}_{n, k},\tau_{n,k})(\omega_{n,k}^{\prime}-\omega_{n,k})+\frac{\zeta}{2}||\omega_{n,k}^{ \prime}-\omega_{n,k}||^{2}\enspace. \tag{13}\]8



where \(n^{\prime}\in\mathbb{N}(n)\) is the neighborhood set of vertex \(n\), \(\theta\) is the parameter of the GNN, \(A_{\theta}\) is an aggregation function, \(C_{\theta}\) denotes the concat operation that combines the aggregated neighbor information with the features of vertex \(n\) itself in \(l-1\) to obtain the new embedding \(h_{n,k}^{(l)}\), and \(l\) is the number of the layers in the GNN. In order to avoid over-fitting during the GNN training process, \(l\) is set to 3.