# A Reinforcement Learning Routing Protocol for UAV Aided Public Safety Networks
## 4 Performance Analysis  
For simulations, we consider an area of \(100\times 100\) m\({}^{2}\) with \(N=100\) nodes is considered as shown in Figure 4 with the UAV placed at the edge and CC placed further away for safety reasons. The UAV and CC are static and their placement allows a limited number of nodes in their transmission range. These nodes (GW and CH) are consequently used to reach the UAV and CC in a multi-hop manner resulting in rapid depletion of their battery. In Figure 4, the blue color is used for CC, the yellow color is used for UAV, the black color is used for CHs, the green color is used for GWs, the red is used for CMs, and the white color is used for Dead Nodes (DNs). The white nodes are not active and unable to communicate due to low or no \(E_{initial}\). This topology is obtained after running NS-3-based simulations for several rounds. The simulation parameters are shown in Table 3.  
Figure 4: Topology of the network in NS-3. For a fair comparison, we include the results of non-clustering schemes based on Dijkstra [53] along with schemes discussed in Section 3.1. To get a better insight into the schemes performance, we assigned random energy varying between 0 and 1 J to all the nodes. Figure 5 shows the number of DNs where the total number of DNs is very high because nodes have random \(E_{initial}\) and they die quickly. The figure shows that both Dijkstra-based schemes had the highest number of DNs. This is approximately 57% higher compared to clustering-energy at 250 s, respectively. Figure 6 shows the residual energy of different schemes. As expected, the Dijkstra-based scheme had the lowest residual energy at 250 s. The Dijkstra-based scheme consumed 61% higher energy as compared with clustering-K mean-GW at 250 s. Figure 7 shows the throughput of all the schemes. Until 90 s, Dijkstra with UAV shows the highest throughput, but after 90 s its curve starts to saturate due to the increasing number of DNs. Clustering-energy-GW has the highest throughput after 250 s i.e., 38 and 40% higher as compared to Dijkstra with UAV and clustering-energy, respectively. Figure 8 shows the EE of different  
Figure 5: Number of dead nodes. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Values** \\ \hline Number of Devices (N) & 100 \\ Network Grid & 100 m \(\times\) 100 m \\ CC Placement & \((120,35)\) m \\ UAV Placement (Initial) & \((100,0,10)\) m \\ Size of Data Packet (L) & 1024 bytes \\ Header Size & 40 bytes \\ Initial Power Level & 0 to 1 J \\ \(E_{Tx}\) & 50 nJ/bit \\ \(E_{Rx}\) & 50 nJ/bit \\ Threshold & 4.35 mJ \\ Cluster Range, \(R_{max}\) & 30 m \\ CH Tx Range, \(T_{max}\) & 45 m \\ Distance b/w UAV and CC & 60 m \\ Max Transmissions in a Round (\(N_{Tx}\)) & 5 \\ \(I_{max}\) & 1024 \\ \(\alpha\) & 0.5 \\ \(\beta\) & 0.5 \\ Discount Factor (\(\gamma\)) & 0.8 \\ Learning Rate (\(V\)) & 0.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Simulation parameters. schemes. The clustering-K mean-GW has highest throughput per unit of energy until 35 s. Beyond 35 s, clustering-K mean shows highest throughput starting from 75 s to 180 s. Both the Dijkstra-based schemes have the lowest EE and decrease further with time. The clustering-K mean has a 29 and 45% higher EE as compared to clustering-energy and Dijkstra without UAV. Figure 9 shows end-to-end delay. The delay is very high in all the schemes because of the high hop count. In comparison, Dijkstra without UAV shows the highest delay followed by Dijkstra with UAV. Clustering-K mean-GW offers a 75% lower delay when compared with Dijkstra without the UAV. Figure 10 shows Packet Delivery Ratio (PDR) of different schemes. In the beginning, both the non-clustering schemes (Dijkstra) have a higher PDR upto 75 s. Afterwards their performance degrades and Dijkstra w/o UAV has the lowest PDR. Compared to (Dijkstra), the PDR of clustering schemes decays slowly. The clustering-energy-GW have the highest PDR after 150 s and it is 9.2% higher than that of clustering-K mean at 250 s.  
Figure 6: Residual energy. Figure 7: Throughput.