# Traffic Light Cycle Configuration of Single Intersection Based on Modified Q-Learning
## 3 Traffic Light Cycle Recommendation Methods
### Road Traffic Indicator  
Traffic congestion is a complicated problem [11]. There are many parameters needed to be considered, such as the Saturation Flow Rate of the intersection, Lane Capacity, Effective Green Light of the Saturation Flow Rate, Degree of Saturation and Average Vehicle Delay Time (AVDT). These data can be calculated by the following methods. Saturation Flow Rate, \(S\), refers to the number of vehicles that pass through an intersection within one hour, and is often used to evaluate the throughput of the lane. The calculation formula of \(S\) is as follows:  
\[S\ =\ \frac{3600}{H} \tag{1}\]  
where \(S\) is the Saturation Flow Rate of the intersection, and it is in units of vehicles/green light hour, and H is the number of seconds it takes for a car to pass through the intersection, and it is in units of second/vehicle. 3600 means that an hour is 3600 s. The maximum lane capacity \(Q_{max}\) can be defined as follows:  
\[Q_{max}=\ \frac{S\times g}{C} \tag{2}\]  
where \(Q_{max}\) is the maximum lane capacity, and it is in units of vehicles/hour, \(g\) is the effective time of the green light, and \(C\) is the time of the cycle, and it is in units of seconds. It is noted that the vehicle will lose partial time due to delay while starting from the stop state (that is defined as loss time). The effective green light is defined as the green light time minus the loss time. The calculation formula is as follows:  
\[g\ =G+Y+R-(L1+L2) \tag{3}\]  
\(G\) is the green light time in a cycle, \(Y\) is the yellow light time in a cycle, \(R\) is the red light time in a cycle, \(L1\) is the loss time when the vehicle starts, \(L2\) is the loss time of the yellow light and the full red light in a cycle. \(L1\) or \(L2\) must be set differently in different regions and road sections. This is the result of long-term observation. In this experiment, \(L1\) and \(L2\) are preset to 4 s. The Degree of Saturation can be known from the above Equation (2), and the calculation formula is as below:  
\[D\ =\ \frac{T}{Q_{max}} \tag{4}\]  
where \(D\) is the degree of saturation, \(T\) is the Traffic flow, which means the total number of vehicles passing through this road in one hour,and it is in units of vehicles/hour, and \(Q_{max}\) is the maximum lane capacity. If \(D\) is greater than 1, it means that the traffic flow has exceeded the lane load, which will cause congestion. On the contrary, if \(D\) is less than 1, the traffic flow does not reach the maximum load of the lane. Through Equation (4), the average vehicle delay time (AVDT) at a traffic intersection can be estimated. The calculation formula is as below:  
\[D_{t}\ =\ \frac{D_{n}*T_{n}+D_{e}*T_{e}+D_{s}*T_{s}+D_{w}*T_{w}}{T_{t}} \tag{5}\]  
where \(D_{t}\) is the average vehicle delay time of the intersection, in units of seconds/vehicle, \(D_{n}\) is the northbound lane average vehicle delay time, in units of seconds/vehicle, \(D_{e}\) is the eastward lane average vehicle delay time, in units of seconds/vehicle, \(D_{s}\) is the southbound lane average vehicle delay time, in units of seconds/vehicle, and \(D_{w}\) is the westward lane average vehicle delay time, in units of second/vehicles. \(T_{n}\) is the northbound lane traffic flow, in units of vehicles/hour, \(T_{s}\) is the eastward lane traffic flow, in units of vehicles/hour, \(T_{t}\) is the southbound lane traffic flow, in units of vehicles/hour, \(T_{w}\) is the westward lane traffic flow, in units of vehicles/hour. If the \(D_{t}\) is larger, the traffic at this intersection becomes more congested. To estimate the efficacy of the model, we calculate the error rate of the AVDT as follows:  
\[E\ =\ \frac{D_{\textit{Qlearning}}\ -\ D_{\textit{best}}}{D_{\textit{best}}}\times 100\% \tag{6}\]  
where \(E\) is the error rate of the AVDT between the exhaustive search and modified Q-learning mechanisms, in units of %, \(D_{\textit{Qlearning}}\) is the modified Q-learning AVDT, in units of seconds/vehicle, and \(D_{\textit{best}}\) is the exhaustive search scheme [32] AVDT, in units of seconds/vehicle.