# Autonomous Intersection Management by Using Reinforcement Learning
## 4 Results and Discussion
### Experimental Results
#### 4.3.1 Evaluation of Training Results of DRLAIM
When we want to deploy the model in the real-time environment, we need to consider the average queue length, average waiting time and throughput with different time step values, such as 0.1, 0.2 and 1 s. Tables 7-9 show the average performance over 10 simulations for the models trained with 4 different number of epochs by using 3 different action time steps. We can see that in-terms of reward performance, the trained model using 100 epochs and a time step of 0.2 s is best. Figures 11-14 show the value of the three performance indicators introduced above, namely AQL, AWT, and throughput, for models trained with four different numbers of epochs using three different action time steps. Figure 11 shows the performance results when training with 0.2, 0.5, and 1 time steps by using 25 epochs. From Figure 11, we can see that it outperforms the model in terms of average waiting time, queue length, and throughput during the time step value 1. However, it is not giving good performances regarding average queue length, average waiting time, and throughput during the time step value of 1 s when the epoch is 50. Figure 12 shows the performance results while training with 0.2, 0.5, and 1 time steps and using 50 epochs. From Figure 12, we can understand that 50 epochs give good performances in terms of average queue length, average waiting time, and throughput during the time step value of 0.2 s.  
Figure 9: Total reward while training using action time step 0.5 s with epochs (**a**) 25, (**b**) 50, (**c**) 100, and (**d**) 200. Figure 11: Performance results while training with three action time steps and using 25 epochs. **(a)** AQL. **(b)** AWT. **(c)** Throughput. Figure 10: Total reward while training using action time step 1 s with epochs **(a)** 25 **(b)** 50 **(c)** 100 **(d)** 200. Figure 13 shows the performance results while training with 0.2, 0.5 and 1 time steps using 100 epochs. We can understand that AWT, AVL and throughput are during the time step value 1, which is depicted in the Figure 13. Figure 14 shows the performance results while training with 0.2, 0.5, and 1 time step and using 200 epochs. From Figure 14 we can conclude that 200 epochs give good performances in terms of average queue length, average waiting time, and throughput during the time step value of 1 s. Therefore, we need to decide on time steps and a number of epochs to deploy the model in the real-time environment. For example Figure 13, we can understand that the proposed model performs well in terms of all the parameters during the time step value 1. Figure 12: Performance results while training with three action time steps and using 50 epochs. (**a**) AQL. (**b**) AWT. (**c**) Throughput. Figure 14: Performance results while training with three action time steps using 200 epochs. (**a**) AQL. (**b**) AWT. (**c**) Throughput. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**EpochsPerformance Indicator** & **Total Reward** & **AQL (Vehicles)** & **AWT (s)** & **Throughput (Vehicles)** \\ \hline
**25** & \(-\)36,076.3 & 69.89 & 44.12 & 125.1 \\ \hline
**50** & \(-\)37,373.7 & 72.04 & 45.69 & 116.8 \\ \hline
**100** & \(-\)33,816.0 & 63.88 & 40.44 & 150.3 \\ \hline
**200** & \(-\)50,636.0 & 63.79 & 44.57 & 144.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Average performance of model trained with different number of epochs using action time step 0.2 s.