# A Reinforcement Learning Routing Protocol for UAV Aided Public Safety Networks
## 4 Performance Analysis
### Combined RL and UAV(s) Trajectory Optimization  
In the previous sections, the UAV was considered to be a static node. However, the UAVs act as flying relays and with an adequately designed flight trajectory they can provide uniform coverage to all the CHs, thus increasing EE. In this paper we consider the placement of single and multiple UAVs and analyze the impact of their trajectory. The  
Figure 14: Control packets sent by different clustering schemes. Figure 13: Control packets sent by all the schemes. Figure 15: Number of CH changes in different clustering schemes. purpose of multiple UAVs is to decrease the number of hops and provide an improved EE. In the disaster scenarios, the energy of the devices is very critical so to further improve EE and connectivity, a multiple-UAV scenario is applied. The speed of UAVs in motion is 2.70 m/s. The transmission range of a UAV is 60 m. If the CHs falls in the transmission range of two UAVs, the CH will choose the UAV closest to the CC. The RL is used to find the best routing paths in the scenario given in Figure 16. Figure 16a provides three dimensional view of the disaster scenario and Figure 16b shows the trajectory paths of UAV's around the disaster scenario. We consider pre-defined UAV flight paths around the area which are discussed below. * **Two UAVs on Opposite Axis and Same direction (TUOAS)**: In this scheme, the two UAVs are placed at the edge of the building. Both the UAVs are placed on the opposite corners of the building. They start moving from the same side of the building, as shown in Figure 16. Both these UAVs are moving in parallel to each other but the direction they are following is the same. The trajectory they are following is along the straight line alongside the building. When they reach the opposite corner of the building, they will follow the same path backwards. The CHs that are in the range of any of these UAVs will send there packets through the respective UAV. Thus, the Equation (15) for the two UAVs will be modified as: \[Cost_{1}=\beta\bigg{(}\alpha\bigg{(}\frac{d_{m_{1}}}{max(d_{m_{1}})}\bigg{)}+( 1-\alpha)\bigg{(}\frac{d_{m_{1}(\text{UAV}_{1})}}{max(d_{m_{1}(\text{UAV}_{1}) })}\bigg{)}\bigg{)}+(1-\beta)\bigg{(}\frac{E_{m_{1}}}{max(E_{m_{1}})}\bigg{)}.\] (17) Here \(Cost_{1}\) is the cost associated with UAV\({}_{1}\) in which \(m_{1}\) and \(n_{1}\) are the sender and receiver nodes associated to the UAV\({}_{1}\): \[Cost_{2}=\beta\bigg{(}\alpha\bigg{(}\frac{d_{m_{2}}}{max(d_{m_{2}})}\bigg{)}+( 1-\alpha)\bigg{(}\frac{d_{m_{1}(\text{UAV}_{1})}}{max(d_{m_{2}(\text{UAV}_{2}) })}\bigg{)}\bigg{)}+(1-\beta)\bigg{(}\frac{E_{m_{2}}}{max(E_{m_{2}})}\bigg{)}.\] (18) Similarly \(Cost_{2}\) is the associated cost with the UAV\({}_{2}\). While \(m_{2}\) and \(n_{2}\) are the sender and receiver nodes associated to the UAV\({}_{2}\). In the case, sender \(m\) is in the direct range of CC, the cost will be calculated using Equation (15). The above equations can also be used for all other two UAV schemes presented below. * **Two UAVs on Opposite Axis and Opposite Direction (TUOAO)**: In this scheme two UAVs are placed at the opposite corner of the building as shown in Figure 16. Both UAVs moves along a straightline alongside the building towards their respective direction. By moving in this way they will help in maximize the coverage area of the building affected by the disaster. When a UAV reaches the edge of the building  
Figure 16: Trajectories of UAVs around the disaster area. it will follow the same path backwards and it keeps on doing this till the end of the simulation. * **Two UAVs moving on Same Axis (TUSA)**: In this scheme, both the UAVs were placed on the same axis separated by 60 m, as shown in Figure 16. Both the UAVs move in the same direction and on reaching there respective endpoint they follow the same path backwards. The maximum separation between them remains the same. * **Single UAV in motion (SU)**: In this scheme, we placed a single UAV at the corner of the building.