# Traffic Signal Control with State-Optimizing Deep Reinforcement Learning and Fuzzy Logic
## 5 Methodology  
The scikit-fuzzy library was used for the fuzzy logic toolkit. Skfuzzy is an alternative name for the software [44]. The chosen parameters are used to determine the inputs GP (green phase) and RP (red phase) [29], which indicate the number of vehicles at the intersection to be granted a green light and the number of vehicles waiting at a red light, respectively. The GP values in this experiment ranged from 0 to 20, whereas the RP values ranged from 0 to 60. The green duration value is the output and ranges from 0 to 30. From this experiment, a dataset was generated that included all conceivable combinations of GP and RP due to fuzzy logic processing, where selecting a new phase requires a time delay of 0.5-1 s, resulting in system latency. In the process, where the time for the green light is determined, the input values GP and RP are used to obtain values from the pre-existing result set instead of repeating the fuzzy logic processes.  
Figure 7: (**a**) Traffic generation over a single episode based on vehicle categories. (**b**) Traffic generation over a single episode based on vehicle direction. (**c**) Traffic generation over a single episode based on intersection direction.  
Fuzzy logic processing uses the Mamdani fuzzy inference method [44]. The first step is fuzzification, which takes an input value that is a precise number, applies a membership function, and adapts it according to the membership level, which is fuzzy data, as shown in Figure 8a,b. A description of the membership functions is presented in Tables 5 and 6. Then, it is sent to the inference engine process using the IF-THEN concept based on the predefined rules presented in Table 7. However, the input can match several rules. Next, the results derived from the inference engine process using all the defined rules are combined using the union operation.  
Defuzzification converts fuzzy sets into crisp values using a membership function, as shown in Figure 9a. A description of the membership functions is presented in Table 8. This method determines the centroid or center of gravity (COG) [44] on the x-axis, with all the results rounded to the nearest integer. An example of the results obtained is the combination of all results per defined rule and the defuzzification process. When GP was set to 7 and RP was set to 3, the center of gravity was located on the x-axis at 12.097. Therefore, the duration of the green light during this phase is 13 s, as shown in Figure 9b.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & \multicolumn{4}{c}{RP} \\ \hline  & vl & l & m & h & vh \\ \hline \multirow{2}{*}{GP} & vl & vvs & vvs & vvs & vvs \\ \cline{2-6}  & vl & vvs & vvs & vvs & vvs \\ \cline{2-6}  & l & s & s & vs & vs & vs \\ \cline{2-6}  & m & m & m & s & s \\ \cline{2-6}  & h & l & m & m & s \\ \cline{2-6}  & vh & vvl & vl & l & m & m \\ \hline \hline \end{tabular}
\end{table}
Table 7: Fuzzy inference rules.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{4}{c}{**Green Duration**} \\ \hline
**Index** & **Membership Function** & **Description** & **Range** \\ \hline
1 & vvs & Very very short & 0–5 \\ \hline
2 & vs & Very short & 0–10 \\ \hline
3 & s & Short & 5–15 \\ \hline
4 & m & Medium & 10–20 \\ \hline
5 & l & Low & 15–25 \\ \hline
6 & vl & Very low & 20–30 \\ \hline
7 & vvl & Very very low & 25 onwards \\ \hline \hline \end{tabular}
\end{table}
Table 8: Description of membership function of green duration.  
Figure 9: (a) The output fuzzy membership of the green duration. (b) Examples of results obtained from the defuzzification process.