## 5 Experimental Setup  
### CARLA Simulation Map  
The simulation environment used in the experiments was based on five urban maps provided by CARLA. Figure 5 illustrates the layout of each town map, while Table 4 summarizes the specific driving environment characteristics of each town. Table 5 presents the durations of the traffic light cycles for each town. As shown in Figure 5, a total of five towns were selected, with four distinct driving episodes assigned to each town. The route length of each episode ranged from a minimum of 150 m to a maximum of 900 m.  
Figure 5: CARLA town maps used in the experiments. In this study, we used all five towns presented in Figure 5, which are publicly available and designed for urban driving scenarios, from the ten towns provided in CARLA version 0.9.13. Town02 features the smallest map among the five towns, characterized by a relatively simple road layout with a limited number of T-shaped intersections and traffic lights. Despite its compact size, it contains a large number of signalized intersections, making it particularly suitable for the initial policy learning of reinforcement learning agents being trained under diverse traffic regulations and geographical variations. In contrast, Town01 is a small urban map that also includes numerous T-intersections and traffic signals, offering a similar baseline environment for early autonomous driving experiments. Town03 includes a small number of cross-shaped intersections and two-lane roads. It is the only town among the five that contains a roundabout and a tunnel, thus providing a more complex setting in which to evaluate the models' driving performance under diverse geographic variations. Town04 features the widest roadways, with four-lane highways, and includes a highway driving scenario. It is used to train and evaluate mixed urban and highway driving strategies, including highway merging, exiting, and lane changes. Town05 incorporates the greatest number of cross-shaped intersections and traffic lights, along with highway and multi-lane roads. It is designed to challenge agents with high-level intersection handling, frequent lane changes, and responsiveness to traffic signals [77]. To evaluate the generalization capabilities of the learned driving policies, training was conducted exclusively in Town02, where the driving route was randomly set for each episode. Table 6 summarizes the maximum route lengths (in meters (m)) of each episode run across the five towns (Town01-Town05) for the training policy. Town01 includes the episode with the longest driving distance, featuring a total route length of 2752 m, and has an average episode length of 688 m, which is more than twice that of the other towns. Despite its relatively simple road structure, the extended route lengths in Town01 increase its difficulty. In contrast, Town02 offers the shortest evaluation \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & **Episode 01** & **Episode 02** & **Episode 03** & **Episode 04** & **Total** & **Mean** \\ \hline Town01 & 930 & 759 & 784 & 279 & 2752 & 688.0 \\ Town02 & 175 & 264 & 313 & 156 & 908 & 227.0 \\ Town03 & 545 & 433 & 341 & 399 & 1718 & 429.5 \\ Town04 & 196 & 352 & 558 & 387 & 1493 & 373.3 \\ Town05 & 418 & 388 & 341 & 340 & 1487 & 371.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Driving distance per episode (m). \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Town** & **Map Size (m)** & **Cross** & **T** & **Round** & **Max Lane** & **Tunnel** & **Traffic Light** & **Highway** \\ \hline Town01 & 342 \(\times\) 413 & 0 & 12 & 0 & 1 & 0 & 36 & No \\ Town02 & 205 \(\times\) 208 & 0 & 9 & 0 & 1 & 0 & 24 & No \\ Town03 & 438 \(\times\) 483 & 5 & 14 & 2 & 2 & 1 & 38 & No \\ Town04 & 816 \(\times\) 914 & 8 & 21 & 0 & 4 & 0 & 43 & Yes \\ Town05 & 430 \(\times\) 486 & 13 & 8 & 0 & 3 & 0 & 54 & Yes \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Driving environment variations across five urban towns.