# Environment-Aware Adaptive Reinforcement Learning-Based Routing for Vehicular Ad Hoc Networks
## 5 Simulation Results and Analysis
### Simulation Environment and Parameters
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Simulation area & 2 km \(\times\) 2 km \\ Number of intersections & 25 \\ Number of segments & 40 \\ Number of vehicles & 400 \\ Signal transmission range & 250 m \\ Packet size & 512 bytes \\ Packet sending rate & 20 \(\sim\) 120 packets/sec \\ MAC protocol & IEEE 802.11p \\ Bit rate & 6 Mbps \\ Path loss exponent & 2 \\ Propagation model & Free-space \\ Beacon interval & 1 s \\ Simulation duration & 400 s \\ \(T_{BW}\) & 1 s \\ \(\alpha_{max},\alpha_{min}\) & 0.95, 0.6 \\ \(\gamma_{max}\) & 0.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameters used in simulations. Figure 8: \(\mathcal{Q}_{max}\) convergence terrain map. In the experiments, it was necessary to determine the variable parameters in the routing algorithm based on the simulation environment. When calculating the received signal-strength factor, \(RSSI_{max}\) and \(RSSI_{min}\) needed to be determined. \(RSSI_{max}\) was determined based on the average statistically calculated RSSI under interference-free conditions at the limit communication distance. \(RSSI_{min}\), on the other hand, needed to be adjusted based on the simulation and was determined empirically. When calculating the link-duration factor, the parameter \(n\) needed to be determined. This parameter was determined by simulating the packet delivery rate, throughput, average hop count, and overhead ratio. From Figure 9, it can be observed that when \(n\) equaled 4, 5, and 6, the packet delivery rate, throughput, and overhead ratio all achieved satisfactory optimal values. However, with the increase in \(n\), the average hop count also gradually increases. Therefore, in this context, \(n=4\) represents the most ideal choice. Consequently, this result was used for all subsequent simulations. This study compared the dynamic and static learning rate of the EARR with the GPSR [13] and CAEB [40], and itanalyzed the simulation results. The following metrics were used to evaluate the performance of the routing protocols:  
* Packet Delivery Ratio: The ratio of the total number of packets received at the destination node to the total number of packets generated by the source node. * Throughput: Throughput refers to the amount of data or the number of data packets successfully transmitted within a unit of time. * Average Hop Count: The average number of nodes that a packet passes through on the routing path from the source node to destination node. * Standardization Delay: The average end-to-end time refers to the time it takes for data to be transmitted from the source node to the destination node along the routing path. To fairly assess the delay experienced by mobile nodes, delay is standardized based on the length of the routing path. * Overhead Ratio: The total number of generated packets minus the numbers of packets successfully received by the destination and then divided by the numbers of packets successfully received by the destination. * CBR Flows and Constant Bit Rate (CBR) traffic: During the simulation process, the source nodes for each CBR flow are initially randomly selected by the program and remain unchanged during the simulation. In this context, CBR flow simulation involves an increase from 10 to 60 nodes, with each node sending four data packets of 512 bytes per second. Figure 9: Performance of different parameters of \(n\). Each data point in the graphs is the average of simulation runs with different random seeds. Each simulation lasted for 400 s. In each simulation, the source node needed a warm-up time before starting to transmit packets and ended transmitting packets 20 s before the end of the simulation.