# Q-LBR: Q-Learning Based Load Balancing Routing for UAV-Assisted VANET
## 5 Simulation Results and Analysis
### Performance Analysis
U2 RV is a multi-criteria routing protocol based on segment density and distance. This protocol only considers the possibility of increasing the traffic through the segment density and does not consider the actual user traffic that may occur in each GVN. In particular, it can be seen that an increase in retransmissions due to a topology change under the same URN coverage may degrade the total network performance. Figure 10: Performance comparison for ground node speed: (**a**) total PDR, (**b**) total network utilization, and (**c**) total latency. Figure 9: Q-LBR versus Q-LBR without Reward Control Function (RCF) for Cumulated Reward Value. Q-LBR (w/o QL) is the result of setting a fixed URPA value (_URPA_upper_ = 60, _URPA_lower_ = 10), except for the Q-learning process. Compared to U2 RV, although there is an improvement in performance owing to traffic distribution, a problem occurs in that it is not possible to increase the utilization of the URN by adapting to changes in the network environment. The resulting latency is compared with that of U2 RV (20 m/s) in Figure 10c. Based on this result, it can be seen that the fixed URPA may not be properly adapted to the network environment under certain situations. By contrast, Q-LBR shows that it can cope with topology changes caused by network mobility through Q-learning. Q-LBR enables the URPA value to be adaptive to the network situation based on the learning process through RCF. As a result, the changing trend of the graph as the speed increases shows a rather gentle curve compared to the other results. In Table 3, Q-LBR shows a lower COP performance than that of U2 RV. This is because COP packets are dropped under congestion or routed only through the ground path by the dynamic URPA. From a system perspective, because COP is a service that is less sensitive to delay and loss, it is reasonable to prioritize USM and RTS. Based on a moving speed of 30 m/s and total traffic flows, Q-LBR shows a PDR of approximately 89.8%, network utilization of 49.1% and latency of 1.27 s.  
Figure 11 and Table 4 show the performance results in terms of the traffic request rate (requests/s), which were similar to those obtained in a previous simulation. However, in the case of a large number of traffic requests exceeding the network capacity, the load balancing efficiency is reduced owing to the multihop resource occupancy of low-priority traffic. This result shows that the dropping of packets in the first hop of the bottleneck link through the URN is more advantageous than dropping through a multihop ground relay. This problem can be solved using the QoS technique (e.g., shaping or policing) to limit the amount of traffic output transmitted with a low priority. Based on the traffic request rate of 30 r/s and the total traffic flows, Q-LBR shows a PDR of approximately 73.6%, a network \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Protocol** & **Speed (m/s)** & **Traffic Type** & **Packet Delivery Ratio (\%)** & **Network Utilization (\%)** & **Latency (s)** \\ \hline U2 RV & 10 & USM & 78.3 & & 1.3 \\  & & RTS & 77.1 & 33.6 & 1.5 \\  & & COP & 79.7 & & 1.6 \\ \hline  & 20 & USM & 65.4 & & 1.5 \\  & & RTS & 69.1 & 27.3 & 1.8 \\  & & COP & 67.3 & & 1.9 \\ \hline  & 30 & USM & 66.7 & & 2.1 \\  & & RTS & 63.5 & 24.3 & 2.7 \\  & & COP & 61.1 & & 2.8 \\ \hline Q-LBR & 10 & USM & 83.1 & & 0.8 \\ (w/o QL) & & RTS & 82.4 & 44.8 & 1.1 \\  & & COP & 50.5 & & 1.2 \\ \hline  & 20 & USM & 81.7 & & 1.4 \\  & & RTS & 78.5 & 42.1 & 1.9 \\  & & COP & 52.1 & & 2.1 \\ \hline  & 30 & USM & 72.1 & & 1.9 \\  & & RTS & 68.5 & 28.6 & 2.1 \\  & & COP & 58.7 & & 2.2 \\ \hline Q-LBR & 10 & USM & 93.3 & & 0.5 \\  & & RTS & 91.1 & 50.3 & 0.8 \\  & & COP & 67.5 & & 1.8 \\ \hline  & 20 & USM & 92.6 & & 0.6 \\  & & RTS & 90.8 & 50.1 & 0.9 \\  & & COP & 62.1 & & 2.3 \\ \hline  & 30 & USM & 92.5 & & 0.8 \\  & & RTS & 89.7 & 49.8 & 0.9 \\  & & COP & 61.8 & & 2.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Simulation results for varying the speed of the nodes (all traffic = 10 r/s).