# Traffic Light Cycle Configuration of Single Intersection Based on Modified Q-Learning
## 3 Traffic Light Cycle Recommendation Methods
### Road Traffic Indicator
If \(E\) is smaller, the modified Q-learning model efficacy is better. We evaluate the system performance with respect to the processing steps as below:  
\[I\ =\ \frac{S_{\textit{best}}-S_{\textit{Qlearning}}}{S_{\textit{Q learning}}}\times 100\% \tag{7}\]  
where \(I\) is the improving rate of the processing steps by the proposed method, in units of %, \(S_{\textit{Qlearning}}\) represents the processing steps using modified Q-Learning, and \(S_{\textit{best}}\) is the processing steps using the exhaustive search method. If \(I\) is larger, the number of processing steps of the proposed method is fewer, and the system performance is better. The congestion level at the intersection is divided into six levels, according to the \(D_{\textit{t}}\) in the Taiwan Highway Capacity Manual [33], as shown in Table 1. Level A is the average vehicle delay time of less than 15 s (i.e., the traffic is light). On the other hand, Level F is the average vehicle delay time that is higher than 80 s (i.e., the traffic is very heavy). This experiment is based on the Road Traffic Sign Marking Line Setting Plan [34] to define a single phase which is recommended to be between 30 s and 120 s.  
This experiment selects one of the most congested intersections in Taipei. The time ranges are from 8:00 to 9:00 and 17:00 to 18:00 at rush hours. Figure 4 shows the traffic flow and steering rate on the Huanhe N. Rd. and Minsheng W. Rd. when people go to work. The east-west bound road is Huanhe N. Rd., and the north-south bound road is Minsheng W. Rd. The total leaving traffic flow of road D is 1995 vehicles, including 1936 vehicles with a straight traffic flow (97.04%), 59 vehicles with a right-turn traffic flow (2.96%) and zero vehicles with a left-turn traffic flow (0%). The total coming traffic flow of road D is 3172 vehicles, including 3126 vehicles with a straight traffic flow from road B, and 46 vehicles with a right-turn traffic flow from road A. In order to prevent traffic congestion caused by a left turn, road D and road B are forbidden to turn left, so the left turn traffic flow is zero. The total leaving traffic flow of road A is 780 vehicles, including 251 vehicles with a straight traffic flow (32.18%), 46 vehicles with a right-turn traffic flow (5.90%) and 483 vehicles with a left-turn traffic flow (61.92%). Similarly, the traffic flows of roads B and C are shown in Figure 4. Because road C is a one-way street, there is no traffic flowing from road C into roads A, B or D. The traffic flow of the Huanhe N. Rd is much larger than that of the Minsheng W. Rd. Therefore, the traffic light cycle configuration must be different from the general time period. \begin{table}
\begin{tabular}{c c} \hline \hline
**Level** & **Average Vehicle Delay Time(s)** \\ \hline A & \(0\leq\text{AVDT}<15\) \\ \hline B & \(15\leq\text{AVDT}<30\) \\ \hline C & \(30\leq\text{AVDT}<45\) \\ \hline D & \(45\leq\text{AVDT}<60\) \\ \hline E & \(60\leq\text{AVDT}<80\) \\ \hline F & \(80\leq\text{AVDT}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Road traffic congestion level. Figure 5 below shows the traffic flow and steering rate on Huanhe N. Rd. and Minsheng W. Rd. when people get off work. The total leaving traffic flow of road D is 1645 vehicles, including 1556 vehicles with a straight traffic flow (94.59%), 89 vehicles with a right-turn traffic flow (5.41%) and zero vehicles with a left-turn traffic flow (0%). The total coming traffic flow of road D is 2554 vehicles, including 2502 vehicles with a straight traffic flow from road B, and 52 vehicles with a right-turn traffic flow from road A. Similarly, Huanhe N. Rd. is not allowed to turn left because Road C is a one-way street. It is necessary to note that the traffic flow between going to work and getting off work at rush hours is different, and therefore the traffic light cycle configuration must also be different. Figure 6 on the next page shows the intersection traffic data of Huanhe N. Rd and Nanjing W. Rd.