# Q-LBR: Q-Learning Based Load Balancing Routing for UAV-Assisted VANET
## 5 Simulation Results and Analysis  
### Simulation Environments  
In this section, we evaluate the performance of the proposed protocol using the network simulator Riverbed Modeler version 18.7. We summarize the detailed information regarding our simulation parameters in Table 2.  
During the simulation, three types of packets are considered: USM, RTS, and COP packets. USM is a traffic type corresponding to the emergency data and control message of a critical service, and is set to EF, the highest packet priority. The size of the USM packet is set to 256 bytes based on an exponential distribution, and the packet interval is set to 10 requests per second (r/s).  
Since the traffic size and request rate follow the exponential distribution \(f_{X}\) with parameter \(\lambda_{s}\) as follows:  
\[f_{X}(x)=\lambda_{s}e^{-\lambda x} \tag{12}\]  
RTS is a traffic type corresponding to a service requiring a certain amount of real-time data using a codec such as a video stream. The priority of the RTS packet is set to AF21, which is the middle priority of the packet. The size of the RTS packet is set to 1500 bytes, and the packet interval is set to 10 r/s. COP is a traffic type corresponding to TCP data, such as FTP, and is set to CS0, the lowest packet priority. The size of the COP packet is set to 256 bytes based on an exponential distribution the same as USM, and the packet interval is set to 10 r/s. To support the QoS requirements for different services, the IEEE 802.11p EDCA mechanism defines four access categories (AC0-AC3) for each channel. We defined AC0 through AC2 for mapping to USM, RTS, and COP services, respectively. The arbitration interframe space (AIFS) is determined according to the mapping relationship for each service. AIFS indicates the idle channel time that must be endured for a transmission opportunity.  
The overall network layout in the Riverbed Modeler is shown in Figure 7. We applied the urban propagation model provided by the Riverbed Modeler when considering the network connectivity from the building attenuation effect. Initially, 11 radio nodes (10 GVNs and one URN) are deployed within a 1000 m \(\times\) 1000 m region. Each GVN is randomly placed, and the random way point (RWP) model is applied as the mobility model.  
Each GVN generates bidirectional USM, RTS, and COP packets, and each GVN establishes a pair with a random destination for three traffic pairs. The URN performs only the relay role and does not generate traffic except for the routing control message. We conducted the simulation 100 times with a 95% confidence interval.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Layers** & **Parameters** & **Settings** \\ \hline PHY & Data Rate & 1 Mbps \\  & Propagation Loss Model & Urban Propagation Model \\  & Coverage Probability (Air-to-Ground) & \(P_{cov}\)[13] \\  & Frequency Band & 5.9 GHz \\ \hline MAC & Protocol & 802.11p \\  & Slot Time & 13 \(\upmu\)s \\  & SIFS & 32 \(\upmu\)s \\  & AIFNSN[AC0:USM/AC1:RTS/AC2:CO] & 2, 3, 6 \\ \hline Network & Hello Interval & 30 s \\  & Active Route Timeout & 90 s \\ \hline Application & USM Traffic (Size/Rate) & Exp. 256 bytes/Exp. 10 rps \\  & RTS Traffic (Size/Rate) & Con. 1500 bytes/Con. 10 rps \\  & COP Traffic (Size/Rate) & Exp. 256 bytes/Exp. 10 rps \\ \hline Q-learning & Learning rate(\(\alpha\)) & 0.3 \\  & Discount Factor(\(\gamma\)) & 0.7 \\  & \(r_{max}\) & 5 \\ \hline UAV & Altitude & 1000 m \\  & Antenna & Omni-directional \\ \hline \hline \end{tabular}
\end{table}
Table 2: Simulation Parameters.