## 4 Performance Evaluation  
We compared the performance of the proposed method with those of other methods. The simulation was conducted using Python, PyTorch, and simulation of urban mobility (SUMO) [24]. We conducted an experiment with the maps of City1 and City2, which were generated using SUMO, as shown in Figure 4. These two cities were extracted from major roads in parts of Seoul to help with considering real urban traffic environments. The length of the road was at a minimum of 150 m and a maximum of 500 m. The figures on the left in Figure 3(a),b show photographs of the considered city, and those on the right show photographs of the edge of the road on the map. Figure 3(a) shows a simple scenario with fewer edges and intersections. This environment had fewer intersections; thus, there were fewer decision zones. As the edges were long and the number of edges was small, fewer environmental factors were required to be considered. Therefore, it was less dynamic because the number of variables was reduced. Figure 3(b) shows a complex scenario with many edges and intersections. Compared with the simple scenario, this environment was very dynamic because of the shorter edge length, several decision zones, and numerous variables. In each scenario, traffic conditions with different MPRs and traffic densities were used to test the proposed method. Figure 3: Framework of the DQN-based vehicle routing control method. The origin and destination of AVs and HVs were randomly set on the edge road of the map. The AVs and HVs were distributed according to a Poisson distribution with different maximum initial speeds of 15 m/s. The motion of the HVs followed the Intelligent Driver Model (IDM) [25], where the maximum deceleration and acceleration for safety purposes was limited to 5 m/s, and the politeness factor was 0. To evaluate the effectiveness of the proposed method, various vehicle arrival rates that corresponded to vehicle density and MPR values, which themselves corresponded with a mixed ratio of AVs and HVs, were used. The proposed method uses DQN, which consists of the following. The neural network of DQN has four fully connected layers: an input layer, two hidden layers, and an output layer. The first and second hidden layers had 150 and 100 neurons, respectively. A Rectified Linear Unit (RELU) activation function was used for both hidden layers. The ReLU function, defined as \(ReLU(x)=max(0,x)\), is one of the most widely used nonlinear activation functions for RL. The remaining parameters used in the simulations are listed in Table 1.  
To evaluate the performance of the _Proposed_ method, we leveraged two methods with the reward parameters [20], which are defined as follows: the combination of driving speed and accumulated driving distance \((ds+ad)\), and the combination of driving speed and the number of vehicles on the road \((ds+nv)\). The method proposed in [19] is referred to \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Number of episodes & 500 \\ Batch size & 32 \\ Learning rate & 0.001 \\ Replay memory size \((\mathcal{D})\) & 10,000 \\ Discount factor of long-term reward \((\gamma)\) & 0.99 \\ Exploration \((\epsilon)\) & \(1\to 0.01\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: List of the hyperparameters. Figure 4: Seoul city map and road edge using SUMO. (**a**) City1, a simple scenario that considers the real traffic urban environment. (**b**) City2, a complex scenario that considers the real traffic urban environment. as _Compare_, where a reward function was designed with a single objective focusing on driving time. Figure 5 shows the effect of weights \(\omega\) on the performance. The weight parameter \(\omega\) was used as the strategic parameter of the reward function, which resulted in a tradeoff between the driving speed and distance of the AVs. To determine the optimal weight \(\omega\) for the driving speed and distance, we evaluated the performance by varying the weight \(\omega\).