# A Reinforcement Learning Routing Protocol for UAV Aided Public Safety Networks
## 2 System Model  
We consider a man-made disaster scenario in which terrorists attack a large building. It is assumed that the normal cellular/wireless infrastructure is either blocked by security forces or destroyed by the terrorists. We further assume the presence of some on-scene devices (from here on called nodes) held by the trapped victims. If provided with an adequate emergency communication network, the information carried by these nodes can yield great insight for the law enforcement agencies. However, it may not be possible for the emergency communication network to provide coverage to all the nodes simultaneously. In this situation, the nodes can cooperatively communicate with each other to form a D2D multi-hop network. To simplify communication between multiple nodes, the devices can form clusters assisted through the D2D network. The UAV and CC are deployed outside the building perimeter to collect the information from these clusters. The clusters within range of CC communicate directly with the CC and the clusters outside the range of the CC communicate with the CC through a UAV. This situation is depicted in Figure 2, in which the UAV (acting as a relay) is placed outside the building to connect the nodes with the CC used by law enforcement agencies. To protect the CC from an ambush by the terrorists, the CC is deployed slightly away from the building. The symbols are mentioned in Table 1.  
Figure 3 shows a two-tier network with \(N\) nodes that are randomly distributed in 100 m \(\times\) 100 m. The nodes are divided into \(i\) clusters and the set of nodes in each cluster is denoted by \(N_{i}\), where \(i=\{1,\ldots,l\}\). Considering a limitation on the transmit power of on-scene devices, the maximum distance between CH and its CM is restricted to \(R_{max}\). Whereas, a CH in this network can communicate with the other CHs over a maximum distance denoted by \(T_{max}\). The pathloss [50] between links separated by distance \(d\) is calculated from:  
\[PL(d)=20\times log_{10}(d)+46.4+20\times log_{10}(f_{c}/5)+12\times n_{w}+17+4 (n_{f}-1) \tag{1}\]  
where the path loss exponent is assumed to be 2, \(f_{c}\) is the carrier frequency, \(n_{w}\) is the number of walls (taken as 1), and \(n_{f}\) is the number of floors (taken as 0) as we assume that the devices are on the same floor.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Name** & **Symbol** & **Name** & **Symbol** \\ \hline Number of nodes & \(N\) & Number of clusters & \(i\) \\ Cluster radius & \(R_{max}\) & CH Tx radius & \(T_{max}\) \\ K value & \(K\) & Average distance of each K & \(W_{k}\) \\ Distance & \(d\) & Transmissions & \(T\) \\ Reward function & \(Rw\) & Learning rate & \(V\) \\ Discount factor & \(\gamma\) & Path loss & \(PL\) \\ Carrier frequency & \(f_{c}\) & Number of walls & \(n_{w}\) \\ Number of floors & \(n_{f}\) & Floor loss & \(F_{l}\) \\ Throughput & \(r\) & Path & \(\rho\) \\ Routing matrix & \(H\) & Packet size & \(L\) \\ Energy Efficiency & EE & Delay & \(\delta\) \\ Transmitting power & \(P_{t}\) & Bandwidth & \(B\) \\ Noise power & \(N_{o}\) & Packet size & \(L\) \\ Beta & \(\beta\) & Alpha & \(\alpha\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Table of symbols.  
Figure 2: Disaster hit area with Unmanned Aerial Vehicle (UAV) and Command Center (CC).