## 6 Simulation Result
### Analysis of Penalty and Failure Factors  
Tables 11 and 12 present a quantitative comparison of the occurrence rates of five penalty types--vehicle stopped, off-track, too fast, red light violation, and collision--across different DRL algorithms and simulated towns. Among all algorithms, the most frequently triggered penalty was vehicle stopped, followed by red light violation. The vehicle stopped penalty was typically induced when a vehicle failed to proceed after a green signal or unnecessarily halted upon detecting a traffic light despite it displaying a go signal. Additionally, in many cases, vehicles misin \begin{table}
\begin{tabular}{l c c} \hline \hline  & **Center Dev Mean** & **Speed Mean** \\ \hline DDPG & 0.41 & 16.69 \\ TD3 & 0.19 & 14.54 \\ SAC & 0.16 & 13.24 \\ TQC & **0.11** & 14.37 \\ CrossQ & 0.18 & 14.73 \\ PPO & 0.20 & 13.52 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Performance comparison of Centerline Deviation Mean and Speed Mean across DRL algorithms over 1 M steps of learning (Bold text indicates the method that achieves the best performance for each metric). terpreted the road structure--particularly in complex or nonlinear layouts--and stopped inappropriately even when a red light was correctly detected. Both TQC and SAC achieved the lowest penalty rates across all categories, indicating that these algorithms effectively learned driving policies that balance both road-rule compliance and trajectory stability. In contrast, CrossQ recorded the highest overall penalty rates, with especially high frequencies in the vehicle stopped (0.69) and too fast (0.29) categories. These shortcomings can be attributed to CrossQ's architectural design. Unlike standard actor-critic frameworks, CrossQ eliminates the use of a target network to enhance sampling efficiency and reduce computational cost, instead employing Batch Renormalization within the critic network. While this design promotes faster convergence and structural simplicity, it comes at the cost of omitting stabilizing mechanisms like target networks, making it vulnerable to \(Q\)-value overestimation and reduced policy robustness in challenging driving environments, such as signalized intersections. Previous research has also reported a tendency for CrossQ to exhibit a higher \(Q\)-function bias--including overestimation--than other algorithms [35], which aligns with the unstable performance and elevated penalty rates observed in our study. Town03 yielded the highest penalty rates among all environments. Specifically, its roundabout sections posed challenges for directional perception, often leading to vehicle stopped penalties. Moreover, off-track and too fast penalties were also quite frequent in Town03, likely due to the increased difficulty of maintaining lane discipline and speed control while navigating complex circular intersections. Town05, characterized by the largest number of signalized intersections, showed the highest incidence of red light violations. Meanwhile, although Town01 has a relatively simple urban layout, its longer routes increased the likelihood of encountering penalty events during extended navigation. In contrast, Town02, which served as the training environment, recorded the lowest penalty rates across all categories. This highlights the importance of focused pretraining in the target deployment domain to improve operational safety and compliance in autonomous driving systems. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **DDPG** & **TD3** & **SAC** & **TQC** & **CrossQ** & **PPO** &
\begin{tabular}{c} **Total** \\ **(Mean)** \\ \end{tabular} \\ \hline Vehicle stopped & 0.63 & 0.18 & 0.18 & 0.07 & 0.69 & 0.37 & 0.35 \\ Off-track & 0.08 & 0.03 & 0.01 & 0.01 & 0.00 & 0.02 & 0.03 \\ Too fast & 0.00 & 0.12 & 0.09 & 0.10 & 0.29 & 0.07 & 0.11 \\ Red light violation & 0.19 & 0.12 & 0.14 & 0.13 & 0.02 & 0.25 & 0.14 \\ Collision & 0.10 & 0.11 & 0.02 & 0.01 & 0.00 & 0.00 & 0.04 \\ \hline Total (mean) & 0.20 & 0.11 & 0.09 & **0.06** & 0.20 & 0.14 & \\ \hline \hline \end{tabular}
\end{table}