## 5 Simulation Results
### Performance of RL-Based Joint Beamforming Scheme  
Figure 6 shows the cumulative received data of a vehicle on the first lane while four vehicles are moving along the different lanes at a speed of 60 km/h. The transmit power of the RSU is 24 dBm. In the proposed joint beamwidth/interval optimization scheme, the received data linearly increases; however, in the other schemes, the received data does not linearly increase. In particular, as the vehicle passes near the midpoint of the road, the performance deteriorates further. The random selection and beam alignment interval optimization schemes showed almost similar performance, being the worst performance. That is, if the beamwidth is randomly selected or fixed, the performance significantly deteriorates. Hence, it is essential to optimize the beamwidth as the vehicle moves. The proposed joint beamwidth/interval optimization scheme outperforms the other schemes because it adjusts both the antenna beamwidth and the beam alignment interval for each time slot, taking future rewards into consideration. However, if the future rewards are not considered, the performance of the proposed scheme would become inferior to the conventional beamwidth optimization scheme. Figure 5: Convergence of the proposed scheme according to the different values of \(\beta\). Figure 7 shows the cumulative link stability time of a vehicle on the first lane while four vehicles are moving along the different lanes at a speed of 60 km/h. The transmit power of the RSU is 24 dBm. In the proposed joint beamwidth/interval optimization scheme, the link stability time linearly increases, but in the other schemes, the link stability time significantly decreases at the midpoint of the road. In the middle of the road close to the RSU, the beamwidth needs to be widened to keep the same beam alignment time at the road edge. Although the antenna gain of a wide beamwidth is smaller than that of a narrow beamwidth, because the path loss is small at points close to the RSU, the data rate does not decrease significantly. Additionally, because the beam alignment time is short for a wide beamwidth, the beam alignment can be performed frequently to keep the main-lobe beamwidth. Hence, when a vehicle passes near the midpoint of the road, the proposed scheme can increase the link stability time and throughput by adjusting both the antenna beamwidth and the beam alignment interval, but other schemes deteriorate the performance because they cannot control the antenna beamwidth and the beam alignment interval simultaneously. In particular, the beam alignment interval optimization scheme with a fixed beamwidth shows the worst performance because the beamwidth cannot be adjusted near the midpoint of the road. Figures 8 and 9 show the average throughput and the average link stability ratio over the entire time that four vehicles are moving on the road. The transmit power of the RSU is 24 dBm. Figure 8 shows the average throughput as the speed of vehicles increases. As the vehicle speed increases, the beam should be frequently aligned due to beam alignment errors, and therefore, the throughput decreases with the vehicle speed. Moreover, as the vehicle speed increases, the time in which the vehicle leaves the road becomes shorter; therefore, the amount of received data from the RSU decreases. Consequently, as the vehicle speed increases, the average throughput decreases for all of the schemes. The proposed joint beamwidth/interval optimization scheme outperforms other schemes. In particular, when the vehicle speed is 90 km/h, the proposed scheme increases the average throughput by about 15.9% and 98.8% in comparison with the conventional beamwidth optimization scheme and joint beamwidth/interval optimization scheme without future reward, respectively. Figure 6: The cumulative received data of the vehicle on the first lane when \(v=60\) km/h. Figure 8 shows the average link stability ratio of a vehicle on the first lane as the speed of four vehicles increases.