# An Intelligent Cluster-Based Routing Scheme in 5G Flying Ad Hoc Networks
## 2 Related Work
For instance, in DQN, the energy consumption of 100 nodes is 15 units, which means there are 5 transmissions and receptions from source to destination UAVs. Similarly, the energy consumption of 1000 nodes is 40 units, which means there are 20 transmissions and receptions of data packet from source to destination UAVs. DQN outperforms RL with at least 20 units lower energy consumption for lower network density (i.e., 100 UAVs), and at least 15 units lower energy consumption for higher network density (i.e., 1000 UAVs). Therefore, DQN increases network lifetime and network stability. This improvement is attributed to the use of DQN at CC to predict the state-action value of a route based on the availability of residual energy and mobility with respect to

Figure 5.2 Convergence of the DQN algorithm with different learning rates. A higher learning rate \(\alpha\) shows a faster convergence as compare to a lower learning rate \(\alpha\). the proximity of a destination UAV. Subsequently, DQN converge to the most favorable route, which has a lower energy consumption as time goes by, across various network planes (i.e., macro-, pico-, and femto-planes). Meanwhile, the random approach has higher energy consumption (i.e., more than 200 units) due to its randomness, while the energy consumption of the DQN, RL, and optimal approaches does not vary considerably for most network densities, particularly from 700-1000 nodes. #### 5.7.2 Effects of Network Density to Link Breakages

The link breakage of a route increases gradually as network density increases as shown in Figure 5.3

DQN outperforms RL with at least 50% lower link breakages at lower network density (i.e., 100 UAVs), and at least 45% lower link breakages at higher network density (i.e., 1000 UAVs). Therefore, DQN increases successful data transmission, contributing to higher network stability and throughput. This improvement is attributed to DNN that learns the mobility pattern of UAVs and uses the action-value function to choose the most favorable route with a longer lifetime based on the movement of UAVs. Meanwhile, the random approach has the highest number of link breakages (i.e., up to 80%) due to higher energy consumption that causes a higher number of dead nodes which increases the rate of link breakages. In contrast, the optimal approach has the lowest number of link breakages (i.e., from 2% to 10% as network density increases) due to the selected route comprised of nodes with lowest mobility. Figure 5.4 Energy consumption increases with respect to network density. DQN achieves lower values as compared to RL and random approaches. Lower energy consumption improves the energy efficiency of a network. #### 5.7.3 Effects of Network Density to Network Lifetime

The network lifetime of a network increases gradually as network density increases as shown in Figure 5.5

DQN and RL performances share almost similar network lifetime at lower network density (i.e., 100 UAVs), and DQN achieves at least 5000 s longer network lifetime at higher network density (i.e., 1000 UAVs). This improvement is attributed to DNN that learns and uses the action-value function to choose the most favorable route with higher residual energy and lower mobility based on node lifetime in order to reduce the overall number of dead nodes. This helps UAVs to perform data transmission over their respective routes for a longer time duration, contributing to higher network lifetime and QoS. Longer network lifetime (i.e., more than 9000 units) can be seen in Figure 5.6 This is because DQN agents learn and converge to their respective favorable route, which increases network lifetime. The network lifetime of the random approach is significantly lower as

Figure 5.7 Network lifetime increases with respect to network density. DQN achieves higher values as compared to RL and random approaches. Higher network lifetime improves QoS and network stability. Figure 5.8 Rate of link breakages increases with respect to network density. DQN achieves lower values as compared to RL and random approaches.