# An Intelligent Cluster-Based Routing Scheme in 5G Flying Ad Hoc Networks
## 2 Related Work
Figure 6: _Cont._The number of link breakages increases with an increasing network density. A higher network density increases the number of hops, and so it increases the energy consumption of data transmission and reception along a route from a source node to a destination node, resulting in a higher number of link breakages. The number of link breakages decreases with increasing learning rate, which helps to select a route with a higher residual energy and stability. When the learning rate is low, the number of link breakages is high because selected routes have UAVs with low residual energy. For the effects of network density on DQN, the learning rate \(\alpha=0.001\) achieves the lowest link breakages. Lower link breakages improve the stability and lifetime of a network. There is a correlation between different performance measures. While an increase in network density improves network lifetime exponentially, it degrades network performance due to an increase in the rate of link breakages and energy consumption. This is because a higher network density increases the number of nodes in a route (or the length of a route), which increases the number of possible actions (i.e., the potential next-hop or intermediate nodes), resulting in an increased energy consumption for data transmission and reception. Similar trend is observed in our investigation with respect to mobility as shown in Figure 7. In Figure 7(a), the network lifetime increases because the energy consumption

Figure 6: Graphical results for the effects of network density on DQN. Learning rate \(\alpha=0.001\) achieves the lowest energy consumption and rate of link breakages, as well as the highest network lifetime. incurred by data transmission and reception reduces due to increased link breakages as the mobility rate increases. Figure 7: Graphical results for the effects of mobility on DQN. Learning rate \(\alpha=0.001\) achieves the lowest energy consumption and rate of link breakages, as well as the highest network lifetime. #### 5.6.3 Convergence of DQN Algorithm

Figure 8 shows the delayed reward of DQN. It shows that the delayed reward converges at approximately 12 after almost 60 episodes when the learning rate is \(\alpha=0.001\), and the delayed reward converges at approximately 8 after almost 100 episodes when the learning rate is \(\alpha=0.0001\). It is worth highlighting that the learning rate \(\alpha\) value may vary based on the underlying application scenarios. At the initial episodes, the delayed reward for both learning rates \(\alpha\) of DQN is unstable; however, as the episode advances, the delayed reward becomes stable. A higher learning rate \(\alpha\) enables an agent \(i\) to converge faster. On the other hand, a smaller learning rate \(\alpha\) can cause a slower convergence (or a longer training time). ### Simulation Results and Discussions

The simulation results and discussions are as follows. #### 5.7.1 Effects of Network Density to Energy Consumption

The energy consumption of a route (i.e., data transmission from a source UAV to a destination UAV) increases gradually as network density increases as shown in Figure 5.0

Energy consumption is based on the energy consumed by an end-to-end route (i.e., from a source node to a destination node). Based on Table 5, there are three assumptions Table 5): (a) the size of data packets is similar, (b) the transmission of a single data packet in each hop consumes two energy units (i.e., joule), and (c) the reception of a single data packet consumes one energy unit. The energy consumption of a route is caused by data transmission from a source UAV to a destination UAV, which increases gradually as network density increases as shown in Figure 5.1 DQN outperforms RL with at least 20 units lower energy consumption when network density is lower (i.e., 100 UAVs), and at least 15 units lower energy consumption when network density is higher (i.e., 1000 UAVs).