# An Intelligent Cluster-Based Routing Scheme in 5G Flying Ad Hoc Networks
## 2 Related Work
Vertical routing uses VCG, which are selected using LET for increased stability and inter-plane communication. We present a use case scenario and show how the network entities presented in Table 3 operate. Consider a cluster member \(CM_{2,1,1}\) in cluster \(C_{2,1}\) establishes a route to cluster member \(CM_{2,2,1}\) in cluster \(C_{2,2}\) in Figure 1. Both UAVs are from the same network plane \(i=2\). There are three possible routes: (a) a seven-hop route \(CM_{2,1,1}-CH_{2,1}-VCG_{2,1,1}-VCG_{1,1,2}-CH_{1,1}-VCG_{1,1,1}-VCG_{2,2,1}-CM_ {2,2,1}\) with inter-plane communication between network planes \(i=2\) (i.e., more stable) and \(i=1\) (i.e., less stable); (b) a five-hop route \(CM_{2,1,1}-CH_{2,1}-CG_{2,1,1}-CG_{2,2,1}-CH_{2,2}-CM_{2,2,1}\) with intra-plane communication in network plane \(i=2\); and (c) a five-hop route \(CM_{2,1,1}-CH_{2,1}-VCG_{2,1,1}-VCG_{3,1,1}-VCG_{2,2,1}-CM_{2,2,1}\) with inter-plane communication between network planes \(i=2\) (i.e., less stable) and \(i=3\) (i.e., more stable). As the third route has lower mobility, it is selected to prolong the route lifetime in order to ensure route stability. As an added advantage, the third route provides traffic offload from the macro-plane and pico-plane, which generally have higher congestion level, to femto-plane, which generally has lower congestion level. Nevertheless, the first route may still be chosen to ensure successful data transmission when the rest of the routes have higher mobility rates and lower residual energy levels. In the proposed approach, the UAV with a higher residual energy level and a lower mobility rate is selected as the next-hop node, therefore it is not mandatory to use a route in femto-plane. Nevertheless, the route in femto-plane is preferred as it provides UAVs with lower mobility. Additionally, it helps to off-load data traffic from an upper plane with a higher congestion level to a lower plane with a lower congestion level. There are trade-offs between various network parameters. For example, femto-plane UAVs have comparatively lower residual energy yet offer a higher stability (i.e., a lower mobility rate). On the other hand, femto-plane is less congested which also helps to increase throughput and reduces packet loss. In our proposed scheme, the decisions of next-hop selection are made in the CC using DQN. There are five reasons in which real-time decisions can be made by nodes, including those carrying real-time data packets, in FANETs with high dynamicity. We segregate these five reasons into two categories, namely the networking aspect and the learning aspect. The networking aspect improves network stability to reduce negative effects to real-time applications as follows:

* The next-hop selection is performed over a clustered network, which has improved network stability. This is because our proposed vertical clustering scheme selects nodes with higher LET to serve as VCGs for communication among different clusters across different network planes. * CHs, which are the distributed entities, make intra-plane decisions to select the next-hop when the source and the destination nodes are from the same network plane. Decisions are made based on the knowledge of the DQN agent. Meanwhile, the DQN agent in CC makes inter-plane decisions to select the next-hop node when the source and the destination nodes are from different network planes. Decisions are based on long-lifetime data (i.e., predictable mobility pattern). Therefore, nodes carrying data can receive forwarding decisions from CHs and CC, while avoiding the delay incurred in receiving forwarding decisions from the CC. * UAV nodes increase connectivity among clusters. This is because they UAV nodes have a large transmission range due to their elevated lookup angle. The learning aspects are as follows:

* The DQN agent embedded in the CC makes decisions based on state-action values, which represents the long-term reward. Specifically, the action with the highest state-action value is selected.