## 5 Simulation Experiments
### Micro Case Study  
This case compares the control effects of a no-control, a pure game, and a reinforcement learning game condition. The scenario comprises a total of six vehicles. The initial state design of the simulation and the vehicle number names are presented in Table 5. In the simulations, the MDV is modeled using the IDM car-following model prior to the game, and the output of each round during the game is determined by the acceleration distribution of different driving styles in the GMM. The MDV's driving style is moderate. In addition, the conflict determination mechanism for the experiment was that the distance between the CAV in the acceleration lane and its MDV behind the left lane was less than 15 m.  
Figure 8 depicts the track, displacement, speed curve, and acceleration curve of the uncontrolled merging process. The merging of the CAV without control is represented by a model derived from SUMO's LC2013. The results indicate that under high traffic densities on the main road, vehicles attempting to merge have difficulty in identifying an insertable gap that meets the safety constraints and consequently stop and wait in the acceleration lane, which can be seen in subgraph (c) speed curve, where the speed of the merging vehicle is equal to zero. Both dense points in Figure 8a and lines with small slopes in Figure 8b also indicate slow speeds. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Vehicle ID** & **L** & **Speed** \\ \hline Af & 70 m & 11 m/s \\ A & 90 m & 10 m/s \\ Ab & 125 m & 9 m/s \\ Bf & 80 m & 11 m/s \\ B & 95 m & 10 m/s \\ Bb & 125 m & 9 m/s \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Vehicle parameters for simulation scenarios. Figure 7: The rounds of gaming with conservative MDVs and unclassified MDVs. The variation in the parameters of the merging scenario based on the game model is shown in Figure 9. The lack of effective guidance regarding the driving strategy between each round of the game, in the absence of external control, resulted in a stalemate situation and sharp speed fluctuations in Figure 9c,d, which led to a lack of comfort and a low level of performance. Eventually, when it threatens the safety of vehicles on the main road, it forces them to slow down significantly, while merging vehicles accelerate sharply to achieve merging. Nevertheless, the pure gaming approach barely achieves a state of coordination compared to the uncontrolled approach; it can be observed that the slope of the displacement curve of CAV A in Figure 9b is always large compared with Figure 8b. This approach avoids the stop-and-wait gap that typically occurs at the end of the acceleration lane, which improves the timeliness of the merging vehicles. However, this comes at a significant cost, not only in terms of the sacrifice of comfort but also in terms of the creation of potential safety hazards. Figure 10 shows the reinforcement learning game modeling the merging process. The processing time of the game is much shorter, which results from the number of trajectory points in Figure 10a compared with only game-based in Figure 10a. Guided by reinforcement learning, the CAV is able to anticipate the MDV's willingness to coordinate to a certain degree and adopt a more aggressive strategy to merge. By analyzing the speed and acceleration curve in Figure 10c,d, it is clear that there is a significant increase in comfort compared to the pure gaming control scenario. In the uncontrolled case, the time for the research subject to pass through the area is 14 s as observed in Figure 8b. However, under the control of the algorithm proposed, the research subject is observed to traverse the area at the 7 th second in Figure 10b. For this case, the CAV's access efficiency was found to have increased by 50%. Figure 8: Variation curves of merging parameters in uncontrolled scenes. (**a**) Trajectory scatter diagram. (**b**) Displacement curve. (**c**) Speed curve. (**d**) Acceleration curve. Figure 10: Variation curves of merging parameters for game models incorporating reinforcement learning.