# Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs
## 5 Simulation and Performance Analysis  
This paper utilized Keras as the platform of deep learning for training the model of the proposed DRL-FIR method. The simulation area of the FANET was set as 1000 m \(\times\) 1000 m, including 45 UAV nodes with randomly generated positions. The packet size and the transmission range of each node were, respectively, set as 512 Byte and 120 m. The available bandwidth, SINR, queue length, and moving speed factors were separately generated randomly, and the generated data obeyed a uniform distribution. The source node and the destination node were randomly selected, and the simulation set so that a node continued to move in the direction of reflection when it moved to the boundary of the region. The simulation parameters including the model-related parameters are summarized in Table 1. Meanwhile, this paper randomly generated a large number of samples (i.e., snapshots) for model training, and adopted the same method to generate the test set to verify the performance of the model (see Figure 3). For each snapshot in the training set, the model was trained 1000 times. To demonstrate the performance of the proposed method, four algorithms were adopted as comparison methods, which are shown below (It should be noted that most of the existing methods belong to the type of central decision-making, or their applicable scenarios are different from the scenarios considered in this paper in topology. Therefore, the comparison methods considered in this paper are variations of the proposed method or the typical distributed routing method.) * FIR-OH: This method considers only the information of one-hop neighboring nodes. Note that the performance of this method can basically correspond to the results of some existing Q-learning based methods, e.g., [20]. * GPSR: A classical distributed routing algorithm proposed in [35]. Many methods have been improved on this method [19], and this method is often used for performance comparison. * W-GPCR: The weighted greedy perimeter coordinator routing method proposed in [36]. We modified this method so that it could be used in this paper. We set this method to consider a Euclidean distance, channel capacity, and link lifetime with the same weight. \begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameter** & **Value** \\ \hline Simulation area & 1000 m \(\times\) 1000 m \\ \hline Node transmission range \(R\) & 120 m \\ \hline Node number & 45 \\ \hline Available bandwidth \(B\) & 1 Mb\(\sim\)5 Mb \\ \hline Packet size & 512 Byte \\ \hline SINR \(\eta\) & 10 dB\(\sim\)40 dB \\ \hline MAC queue length \(L\) & 1 Kb\(\sim\)10 Kb \\ \hline Moving speed \(L\) & 3 m/s\(\sim\)10 m/s \\ \hline \(\gamma\), \(\epsilon\), \(\alpha\) & 0.9, 0.6, 0.01 \\ \hline Decay rate of \(\epsilon\), \(\epsilon_{decay}\) & 0.995 \\ \hline The minimal value of \(\epsilon\), \(\epsilon_{\min}\) & 0.001 \\ \hline Experience-replay memory capacity & 200 \\ \hline Target network update frequency \(F_{t}\) & 500 \\ \hline Experience-replay minibatch size & 32 \\ \hline \(\mu_{1}\), \(\mu_{2}\), \(\mu_{3}\), \(\mu_{4}\) & 0.25, 0.25, 0.2, 0.3 \\ \hline \(K\), \(Z\), \(Z_{s}\) & 5000, 3000, 600 \\ \hline Training/test set size & 2000/500 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation parameters. Figure 3: Schematic diagram of using snapshots to train and test the model of the proposed method. \(N_{train}=2000\) and \(N_{test}=500\), respectively, denote training set size and test set size in the simulation. * FIR-CD: a variant of the proposed DRL-FIR method. This method only considers the channel capacity and distance factors in Equations (11)\(\sim\)(14). Ignoring factors such as MAC queue length and link lifetime may cause the method to not be well applicable to the dynamics of FANETs. * FIR-DLT: a variant of the proposed DRL-FIR method. This method only considers the distance, MAC queue length, and link lifetime factors in Equations (13)\(\sim\)(19).