# D2D-Assisted Adaptive Federated Learning in Energy-Constrained Edge Computing
## 4 Partite Graph-Based FL Scheme with Adaptive D2D Pairing and Aperiodic Local Iterations
```
0: Initial graph \(G(\mathcal{V},\mathcal{E},\mathcal{X}_{p})\), actor network \(\pi_{\theta}\), critic network \(v_{\phi}\), and clipping ratio \(\epsilon\);
1:for episode=1 to \(epi^{\max}\)do
2: Resets \(s_{0}\) and set reward \(r=0\);
3:for\(k=\{1,2,...,K\}\)do
4: Get action \(a_{k}\) according to (37);
5:if\(r_{n,k}=m\)then
6:\(r_{n,m,k}=1\);
7:endif
8:if\(r_{n,k}=1\)then
9: Randomly chosen \(\tau_{n,k}\) from \([\frac{(E_{n}^{max}-\sum_{i=1}^{k}E_{n,i})/D_{n,k}\epsilon_{n}}{K},\frac{(2K-2 +2H)}{(K-1+2H)\sum_{i=1}^{k}\tau_{n,i}/K}]\) following a continuous uniform distribution;
10:else
11:\(\tau_{n,k}=0\), node \(n\) does not perform model training in the \(k\)th round;
12:endif
13: Get reward \(r_{k}\widehat{A_{k}}=r_{k}\), and turn to next state \(s_{k+1}\);
14: Put \(<s_{k},a_{k},r_{k},s_{k+1}>\) into Replay buffer;
15:if\(T^{\max}-\sum_{i=0}^{k}T_{i}>0\) & \(E_{n}^{\max}-\sum_{i=0}^{k}E_{n,i}>0\)then
16: break;
17:endif
18: Randomly obtain \(I\) records from the replay buffer;
19: Leverage the Adam optimizer to update parameter \(\theta\) of the actor network and \(\phi\) of the critic network, \(\{\theta,\phi\}=\arg\max(Loss(\theta,\phi))\) according to (39);
20:endfor
21:endfor
```

**Algorithm 2** GNN-and-DRL-based D2D pairing algorithm The computational complexity of Algorithm 2 derives from several key parts: policy evaluation (Critic), policy update (Actor), experience replay and sampling, and the FL rounds. In each time step, the Critic network computes the state value or action value through a forward pass. The Critic network is an \(L\) layer GNN, and the forward pass and backpropagation complexity is \(\mathcal{O}(2L\cdot v_{\phi})\), where the vertex features have two dimensions and the network has \(v_{\phi}\) parameters. The Actor network computes the policy through a forward pass in each time step using an \(L\) layer GNN. The complexity of the forward pass and updating parameters is \(\mathcal{O}(2L\pi_{\theta})\), where the actor network has \(\pi_{\theta}\) parameters. Storing each time step's state, action, reward, and next state into the experience replay buffer has a complexity of \(\mathcal{O}(1)\). Sampling a mini-batch from the experience replay buffer for training has a complexity of \(\mathcal{O}(B)\), where \(B\) denotes the batch size. The algorithm runs for \(K\) FL rounds and \(epi^{\max}\) episodes, so the overall complexity is \(\mathcal{O}(epi^{\max}K(2 Lv_{\phi}+2L\pi_{\theta}+B))\). It is noteworthy that Algorithm 2 can be applied to large-scale network scenarios. In a sense, this algorithm can be taken as a server clustering approach, in which the relay servers act as cluster heads while the client servers act as cluster members. The models trained by cluster members are distributively aggregated at their corresponding cluster heads and then delivered to the aggregation server. When the number of servers in the network increases, more servers will be selected as cluster heads to share the model aggregation load of the aggregation server. ## 6 Numerical Results

In this section, we evaluate the performance of our proposed D2D-assisted adaptive FL schemes through experimental simulations. We consider an FL-enabled edge network that consists of 30 edge servers randomly deployed in a rectangular area with a side length of 500 m, together with an aggregation server located within the center of the area. The edge network scenario is shown in Figure 3. The energy budget and the energy cost for processing a unit-sized dataset of the servers are randomly selected from \((10,40)\) and \((0.0004,0.0010)\) following continuous uniform distributions, respectively. We use the MNIST and CIFAR-10 datasets to evaluate our proposed schemes. MNIST is a large database of handwritten digits and CIFAR-10 is a collection of 60,000 32 \(\times\) 32 color images, which are commonly used for training various image-processing systems and have been widely leveraged for the performance evaluation of FL schemes [35].