## 4 Experiments and Results  
### Simulation Scenarios  
This method applies the DRL agents to represent entire vehicles under mixed-traffic conditions in the urban network. In every time step simulation, the DRL agents obtain the updated state information and respond to the new state of the simulation environment in under 0.1 s. In addition, the right-of-way rule, which avoids traffic collisions based on traffic regulations, is used for the vehicle controller. More importantly, DRL agents learn to achieve a higher reward as soon as possible. Continuous routing is used to keep entire vehicles in the urban network. The IDM model is applied to control HVs. Simulation is increasingly being used in autonomous driving control as an excellent chance for potentiality assessment. A simulation works to achieve the best interaction of autonomous vehicles under a mixed-traffic condition, by using some assumptions to make \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Number of training iterations & 200 \\ Time horizon per training iteration & 6000 \\ Hidden layers & \(256\times 256\times 256\) \\ GAE Lambda & 1.0 \\ Clip parameter & 0.2 \\ Step size & \(5\times 10^{4}\) \\ Value function clip parameter & \(10\times 10^{3}\) \\ Number of SGD iterations & 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: A set of hyperparameters for the mixed-traffic conditions in the urban network. SGD: stochastic gradient descent; GAE: generalized advantage estimation. the experience seem as real as possible. But in reality, they are very different scenarios with various disturbance factors. For instance, a vehicle could suddenly brake or cut in another lane, or aggressive drivers could cause accidents on the road. Furthermore, the traffic flow could be affected by overcrowding on the road, violation of the traffic rules, etc. Hence, the simulation could not fully cover the real scenarios due to the limitation of the knowledge and software. In this work, we assumed some simulation conditions, such as platooning vehicles only drive straight ahead to approach the urban network within the real traffic volume. Furthermore, this study tried to achieve a higher average velocity while penalizing traffic collisions among vehicles in the urban network. We conducted various experiments with a time step of 0.1 s, a lane width of 3.2 m, two lanes in each direction, a length in each direction of 3000 m, a distance between two intersections of 400 m, a maximum acceleration of 3 m/s\({}^{2}\), a minimum acceleration of \(-\)3 m/s\({}^{2}\), a maximum speed of 12 m/s, and a traffic volume of 1000 vehicles per hour in each direction. Figure 5 shows the leading autonomous vehicle experiment in the urban network. Figure 5: Leading autonomous vehicle experiments for the urban network: (**a**) mixed-traffic environment under autonomous vehicle (AV) penetration rates ranging from 20% to 80% in 20% increments; (**b**) full autonomy traffic with a 100% AV penetration rate. To exhibit the advantage of the leading autonomous vehicle experiment (the proposed experiment), other experiments are conducted to be compared with the proposed experiment--namely, the leading manual vehicle and entire manual vehicle experiments. Figure 6 shows a comparison of experiments in the urban network. Figure 6: A comparison of experiments in the urban network: (**a**) the entire manual vehicle experiment with a 0% autonomous vehicle (AV) penetration rate; (**b**) the leading manual vehicle experiment under AV penetration rates ranging from 20% to 80% in 20% increments. To evaluate the superiority of the PPO with a clipped objective hyperparameter (the proposed hyperparameter), the PPO with an adaptive KL penalty is compared to the proposed hyperparameter. In particular, the PPO with an adaptive KL penalty updates the policy through the weight control coefficient. This process bases on the difference between the current KL divergence and the target KL divergence [38].