# An Intelligent Cluster-Based Routing Scheme in 5G Flying Ad Hoc Networks
## 2 Related Work
Longer network lifetime (i.e., more than 16,000 units) can be seen for the optimal approach at higher mobility rate in Figure 4:3 This is because DQN agents learn and converge to their respective favorable routes, which increases network lifetime. The network lifetime of the random approach is significantly lower as compared to DQN, RL, and optimal approaches due to its randomness. A higher network lifetime provides stability, contributing to a higher throughput and a reduced signaling overhead caused by route formation. DQN improves network lifetime by up to 60% and 120% compared to RL and random approaches, respectively, and so it increases network stability. Figure 4:4 Rate of link breakages increases with respect to node mobility. DQN achieves lower values as compared to RL and random approaches. Lower rate of link breakages improves QoS and network lifetime. ### Complexity Analysis

We investigate the computational, message and storage complexities of the DQN algorithm. We have taken the inspiration from [60] for complexity analysis. The complexity has different levels: (a) algorithm-wise for a single iteration in the execution of the DQN algorithm, (b) agent-wise all possible state-action pairs at the agent level, and (c) network-wide at the network level. The parameters for complexity analysis are shown in Figure 4:5

Computational complexity defines the number of execution cycles required to predict the state-action value for all state-action pairs of the DQN agents. In DQN (see Algorithm 1), the computational complexity of training is \(O|C_{t}|=O(\sum_{t=2}^{X}C_{t})\), where \(O(C_{t})\) represents the complexity of a single iteration of training at time \(t\in\{1,2\ldots,X\}\) (see steps 6 to 16 in Algorithm 1). The algorithm-wise computational complexity is \(O(|A||C_{t}|)\), which is incurred whenever an agent \(i\) updates network parameters to achieve the desired target function upon receiving a state-action value, and each state has \(|A|\) actions (see step 14 in Algorithm 1). The agent-wise complexity is \(O(|S||A||C|)\) since an agent \(i\) updates its network parameters all the state-action pairs. The network-wide complexity is \(O(|I||S||A||C|)\) in a network with \(|I|\) agents. Meanwhile, in the traditional RL approach (see Algorithm 2), the algorithm-wise complexity is \(O(|A|)\), which is incurred when an agent \(i\) updates its Q-value upon receiving a delayed reward and each state has \(|A|\) actions (see step 9 in Algorithm 2) The agent-wise complexity is \(O(|S||A|)\) since an agent \(i\) updates its Q-value for all state-action pairs. The network-wide complexity is \(O(|I||S||A|)\) in a network with \(|I|\) agents. Message complexity defines the number of messages exchanged among the agents to update a state-action value. In DQN (see Algorithm 1), the algorithm-wise message \begin{table}
\begin{tabular}{l l} \hline \hline
**Parameter** & **Description** \\ \hline \(|S|\) & Number of states. \\ \(|A|\) & Number of actions for each state. \\ \(|R|\) & Number of rewards for each state-action pair \((s_{t},a_{t})\). \\ \(|I|\) & Number of agents in a network. \\ \(|I|\) & Number of neighboring agents of an agent in a network. \\ \(|C|\) & Training complexity. \\ \(|H|\) & Hidden layer complexity. \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Parameters for complexity analysis. Figure 14: Network lifetime increases with respect to mobility rate. DQN achieves higher values as compared to RL and random approaches. Higher network lifetime improves QoS and network stability. complexity is \(\leq|J|\) (see step 16 in Algorithm 1), which is incurred whenever an agent \(i\) exchanges shared local and global information with CC and DCs. Similarly, the agent-wise complexity is \(\leq|J|\). The network-wide complexity is \(\leq|I||J|\) in a network with \(|I|\) agents. Meanwhile, the traditional RL approach and DQN have similar algorithm-wise, agent-wise, and network-wise message complexities (see step 10 in Algorithm 2).