# Traffic Signal Control with State-Optimizing Deep Reinforcement Learning and Fuzzy Logic
## 3 System Design
### State  
This is one of the main components of the proposed method. The current state of the environment is the knowledge used to improve the DQN decision-making process.  
For the system to have knowledge about controlling the traffic flow to improve its effectiveness, sufficient information regarding vehicle data at the intersection must be collected. This state information is necessary data that can be immediately provided to the traffic signal operators by considering the location of the vehicle in the environment.  
The state variable stores data on the waiting time, divided into ten intervals. These data can be used to indicate the vehicle's position and time spent waiting. Variable s\({}_{\text{t}}\) represents the state of the intersection at a specific time step t.  
A division of lanes at traffic intersections was implemented. Each lane was divided into parts known as cells. The cell sizes are designed to be unequal, in which the cell nearest to the intersection is narrow and widens when it moves further from the intersection [20]. This is because of its superior performance compared with the use of a uniform cell size for all distances [29].  
If a cell has a vehicle and there is no waiting time, the value is 1. However, if the waiting time is not zero, the value is divided into ten intervals according to the waiting time, as shown in Table 1, and the cell has a zero value in the absence of a vehicle. Using the state design of the proposed method, it is possible to observe both the presence of vehicles and waiting time. This differs from the prototype Deep Q-network with fuzzy logic [29], which has values of 0 and 1, indicating the presence or absence of a vehicle.  
There were ten cells in each lane. The three lanes on the right were combined into a group unit, and the same traffic signals were used. Therefore, the value recorded in the cell was selected from the maximum values of the three lanes.  
Furthermore, in the proposed method, the length of each vehicle type varies and the size of the cells near the intersection is narrow, which means that a vehicle whose length is greater than the size of the cell can be in the next cell. Consequently, the highest value in the cell is saved to state storage. As shown in Figure 4, this differs from a prototype Deep Q-network with fuzzy logic [29], which has only one type of vehicle, and the length of the vehicle is less than that of the cell.  
Figure 3: Ambulance detection distance in the ambulance prioritization system.  
\begin{table}
\begin{tabular}{c c} \hline
**Level** & **Waiting Time (s)** \\ \hline
1 & 0 \\
2 & 1–6 \\
3 & 7–12 \\
4 & 13–18 \\
5 & 19–24 \\
6 & 25–30 \\
7 & 31–36 \\
8 & 37–42 \\
9 & 43–48 \\
10 & More than 48 \\ \hline \end{tabular}
\end{table}
Table 1: The level of division is determined by waiting time.  
Figure 4: (**a**) Overview of traffic at intersections. (**b**) Overview of the traffic situation on the southern side of the intersection, divided into cells. (**c**) Each car’s waiting time on the southern side of the intersection is divided into cells. (**d**) The stored state values located on the southern side of the intersection.  
Figure 4 displays an example of a state for the southern side of the intersection. Meanwhile, Figure 4c presents the waiting time of each car, divided into cells based on Figure 4b. If a cell contains more than one vehicle, the highest value within the cell is used for comparison in Table 1 before being stored as part of the state, as depicted in cell 3 of Figure 4c,d.